My bachelor's thesis at ETH Zurich
BitThief is a free loading BitTorrent client, that is, it downloads from the swarm without uploading. As a part of this project, following goals were achieved.
- Integrate meta-search engine
- No hard-coding i.e. make system fully user configurable
- Allow users to filter results using query strings
- Show latest torrents on user's chosen categories such as movies, TV, etc.
JetStor NAS 724uxd 724uxd 10g - technical presentationGene Leyzarovich
The JetStor NAS 724UXD is a unified / hybrid NAS storage system that consolidates NAS and IP-based iSCSI SAN in one chassis. Featuring the newest Intel Haswell platform to lower power consumption and 7x 1Gb Ethernet host ports per controller, all encompassed in a small 4U enclosure. The JetStor NAS 724UXD offers SSD Caching to boost random I/O intensive application, Snapshot, Thin Provisioning, Online Capacity Expansion and Controller-based cable-less design for excellent manageability.
JetStor NAS 724uxd 724uxd 10g - technical presentationGene Leyzarovich
The JetStor NAS 724UXD is a unified / hybrid NAS storage system that consolidates NAS and IP-based iSCSI SAN in one chassis. Featuring the newest Intel Haswell platform to lower power consumption and 7x 1Gb Ethernet host ports per controller, all encompassed in a small 4U enclosure. The JetStor NAS 724UXD offers SSD Caching to boost random I/O intensive application, Snapshot, Thin Provisioning, Online Capacity Expansion and Controller-based cable-less design for excellent manageability.
Presentation of my bachelor's thesis at ETH Zurich
BitThief is a free loading BitTorrent client, that is, it downloads from the swarm without uploading. As a part of this project, following goals were achieved.
- Integrate meta-search engine
- No hard-coding i.e. make system fully user configurable
- Allow users to filter results using query strings
- Show latest torrents on user's chosen categories such as movies, TV, etc.
This is brief presentation on BitTorrent technology.
(caution: avoid using site mentioned in the ppt for downloading torrent file.. b t jun kie ....it may not be safe..)
(Thanks to Soumya and my other colleagues for the help.)
Presentation of my bachelor's thesis at ETH Zurich
BitThief is a free loading BitTorrent client, that is, it downloads from the swarm without uploading. As a part of this project, following goals were achieved.
- Integrate meta-search engine
- No hard-coding i.e. make system fully user configurable
- Allow users to filter results using query strings
- Show latest torrents on user's chosen categories such as movies, TV, etc.
This is brief presentation on BitTorrent technology.
(caution: avoid using site mentioned in the ppt for downloading torrent file.. b t jun kie ....it may not be safe..)
(Thanks to Soumya and my other colleagues for the help.)
BitTorrent is the name of a peer-to-peer (P2P) file distribution protocol, and is
the name of a free software implementation of that protocol. The protocol was originally designed and created by programmer Bram Cohen, and is now maintained by BitTorrent Inc. BitTorrent is designed to distribute large amounts of data widely without incurring the corresponding consumption in costly server and bandwidth resources. Cache Logic suggests that BitTorrent traffic accounts for 55% of all traffic on the Internet, while other sources are skeptical. The original BitTorrent client was written in Python. Its source code, as of version has been released under the BitTorrent Open Source License, which is a modified version of the Jabber Open Source License. There are numerous compatible clients, written in a variety of programming languages, and running on a variety of computing platforms.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Crescat
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
BitThief Report
1.
Distributed Computing Group, ETH Zurich
BitThief New UX
Semester Thesis
Vishrant Vasavada
v.vasavada@mail.utoronto.ca
Department of Computer Science,
University of Toronto
Supervisors:
Michael König
Prof. Roger Wattenhofer
2.
Acknowledgements
I would like to express my gratitude, especially to my mentor Michael König for supporting me during
the whole project with a constant input of new ideas and improvement proposals and also for providing
all the assistance needed when I was stuck. I would also like to thank Prof. Roger Wattenhofer who gave
me the opportunity to be the part of this project. Lastly, I would like to thank Prof. Francois Pitt and Prof.
Christiana Amza at University of Toronto who agreed to read my project report and feedback given by
Prof. Wattenhofer, and allowed me to pursue this project.
3.
Abstract
User should be able to have one stop destination for torrent file search, information retrieval (such as
number of seeders, file size, upload date) and download, rather than visiting multiple torrent websites to
obtain the torrent files they want and transfer them to the BitTorrent client. With the BitThief’s (a
BitTorrent client) new User Experience (UX), we try to make this possible by providing the user with a
fully configurable system, integrated into the BitThief, which can act as a metasearch engine, along
with result filters, and take their torrent search, information retrieval and download experience to a
whole new level.
4.
Introduction
BitThief is a “freeloading” BitTorrent client which downloads files from BitTorrent swarms without
returning the favor by uploading. The goal of this project was to implement a flexible userconfigurable
system for integrating existing torrent websites like piratebay.se and kat.to into the BitThief. This would
then allow the users to add, edit or delete any torrent website configurations and get the torrent search
results from the sites, which integrates a metasearch engine into the BitThief in a unique way.
The main goal of this project was to emphasize the concepts of “flexibility” and “power user” which
will be discussed later in this report. While the BitThief already had features of any other normal torrent
client such as downloading from magnet links and viewing some upload/download statistics, now it also
gives its users the ability to perform some amazing operations like executing queries on search results,
creating profiles to autopropose the download of newly created torrents on the pages configured by the
user and retrieve any information about a torrent from its torrent page as per the user configurations.
The purpose of this report is to further explain all these new flexible features that were added to the
client, why they were added and how they can enhance the user experience.
5.
Background
MOTIVATION
Many websites are out there which host links to torrent files and the cached information like IMDB
rating, number of files, file sizes, torrent type and such other details. Whenever a user executes a search
on a torrent website, they often find themselves running through the list of search results while giving
special attention to the specifics like number of seeders, number of times the file has been downloaded,
whether it was uploaded by a verified user, and many such things. Also, some torrent websites might
have some torrent type (e.g. movies) categorized better and more informatively than the others (e.g.
games). Sometimes, a user might be interested in new or “hot” torrents listed on torrent websites, but
only in torrents with some specifics (e.g. size greater than 1 GB or torrent title that contains “YIFY”).
The torrent websites are also often unstable with their domains and their website layouts. All these
would be frustrating and lacking in details as desired by a power user who wants to save time, get only
specific results, automate certain functions and have flexibility. These features are not offered by the
torrent websites and most of the torrent clients today. Absence of these features in the existing torrent
websites and clients and the needs of a power user were a good motivation to come up with a client that
serves as a good search engine, allows users to filter the search results based on queries and also allows
users to update torrent websites’ URLs as needed. What makes BitThief’s new system unique is that it
is a fullyconfigurable system with nothing hardcoded. If a certain torrent website is down or its web
content layout changes, users can simply reconfigure BitThief without requiring its developers to release
a fixed version.
RELATED WORK
Torrentz.eu is a webbased metasearch engine which indexes torrent results from major torrent
websites. “To perform a search, users simply type in the string of keywords and the list of matching
torrent files is displayed on screen for the user to choose from.” It also allows users to sort the results 1
based on seeders and good or verified uploaders. Selecting the torrent from the result list, then takes the
user to another page, listing the torrent websites currently hosting the specified torrent through which a
user can view more details and download the file.
1
Source: https://en.wikipedia.org/wiki/Torrentz
6.
Figure 1: Torrentz displaying the list of external websites hosting the torrent file links 2
Though it automatically keeps track of which torrent website is down or has its domain name changed, it
actually isn’t providing its users with any other information on torrents. Every time the user performs the
query, they have to go through 3 steps to get to the host torrent website and view more information.
One of the most popular clients, ᵰTorrent, does have a torrent search feature where users are allowed to
manage the search providers by entering the search query URLs. Using this feature, users can add,
delete or edit a search engine anytime. This prevents the hardcoding of the search engines.
Figure 2: μTorrent’s Search Engines Manager 3
However, it lacks most of the other useful features which were developed in this project and were
discussed above in brief.
2
Source: https://chrome.google.com/webstore/detail/torrentzeumagnetizer/kcagpojojhhdoiphfpmipackjjchlboj
3
Source: http://www.techsupportalert.com/%20utorrenthelpusingutorrenttorrentsearch
7.
Design and features
We initially started with an idea to integrate the support for finding and filtering new movie torrents
based on the information obtained from IMDB into the existing client. The idea was to retrieve new
releases and the popular movies list from IMDB, based on the user settings like minimum IMBD rating,
movie genre, and other such data, and perform a metasearch on different torrent websites’ search
engines to get related torrents. Two basic input fields were used to reach this goal. The user had to
configure the torrent website URL and the search query URL format for the websites they wanted, like
ᵰTorrent.
The shortcomings to this approach were that there are many types of torrent files available other than
movies and that there is no reliable API similar to IMBD available for most of these types which would
limit our client to only a few torrent categories. Also, such APIs do not tell us anything about the torrent
files that the client retrieves, which users might be interested in, such as number of seeders, quality of
the torrents’ contents, size of the torrent files and other such data.
We had to come up with another way of retrieving the torrent information from the torrent download
page itself. The simple solution to this was to use XPaths, which further opened many other doors for
flexibility. The user is simply allowed to enter the XPaths to all the pieces of information they want our
client to retrieve from the torrent page. Further, the user may enter queries to filter the results and only
display the content they care about.
The backbone behind being able to do all of these lies in our Search Engines Manager tool and the Query
Strings.
QUERY STRINGS
A Query String is simply a statement that evaluates to true or false. It is used to filter the torrent results
retrieved from the torrent websites. BitThief supports following operators in a Query String:
is (e.g. title is inception)
isnot (e.g. title isnot inception)
contains (e.g. title contains harry)
notcontains (e.g. title notcontains harry)
8.
before (e.g. uploaded before “03 december 2015”)
after (e.g. uploaded after “03 december 2015”)
on (e.g. uploaded on “03 december 2015”)
Logic symbols (==, >=, >, <=, <, !=) (e.g. seeders > 300)
It is also possible to group multiple expressions or query statements. For example, uploaded:year <
2016 AND (size < 3 OR size > 500) . This will simply return all the torrents which were
uploaded before the year 2016 and whose size is either less than 3 units or greater than 500 units.
SEARCH ENGINES
Figure 3 shows the Search Engines Manager.
Figure 3: BitThief’s Search Engines Manager
A search engine has following fields:
Torrent Website Base URL (e.g. https://thepiratebay.org): This is simply the torrent website
URL
Torrent Query Format (e.g. https://thepiratebay.org/search/<query>/<index>/7/): This is the
torrent query result website URL format. To retrieve this, the user needs to go to the site, do a
search, replace the search term with “<query>”, and first page number with “<index>” and copy
the resulting string to the client.
These first two fields are enough for carrying out the normal search.
Query Result Page Start Index : Since the number of results would be big most of the time, and
all results can’t be displayed on the first page, our results retriever will need a first page index
(usually 0 or 1) to automatically jump to the next page after it is done browsing through the
current page.
9.
Search Results XPaths : This field is XPath provided by the user to get a list of hyperlinks to all
the results on the page.
Details Page XPaths : By “Details Page” we mean torrent file download page. This is the page
which contains a magnet link, or .torrent file link and other information like seeders, upload date,
torrent size and other such data depending on the website.
Figure 4: BitSnoop’s Torrent Download Page
As shown in figure 4 above, BitSnoop’s details page contains some useful information that a
user might be interested in. Users can simply enter the XPaths to these fields and name them in
their client. These names and information obtained from the XPaths can then be used to perform
simple queries (e.g. Title excludes “HDCam” AND (Seeders > 200 OR Size
contains “GiB”) ) to filter the search results. One key element added to the XPaths was the
support for the regular expressions. As you can see in the figure 4, the file size mentioned is
“1.48 GB”. The XPath might give us the complete value, but to have a query with number
comparisons, the user needs a way to extract the precise data (“1.48”) from the string result
obtained using the XPath.
10.
Figure 5: BitThief’s Details Page XPath Manager
Torrent Categories : As mentioned before, a torrent website can have multiple torrent
categories (e.g., movies, games). Since we were interested in making the BitThief a one stop
destination for everything, we needed a way to update its users with newly available torrents in
their chosen categories. For this, we came up with the concept of “Profiles” and “Torrent
Categories”.
Torrent Categories maintain a list of category names and URLs (e.g. https://kat.cr/movies) and is
linked to a search engine.
Figure 6: BitThief’s Torrent Categories Manager
11.
A profile is simply like a “saved search” containing such categories and query strings that runs
every time the BitThief is opened if the profile is active and displays the new available torrents
in the chosen categories which match the queries.
Figure 7: BitThief’s Profile Manager
SEARCH RESULTS
Based on the query string and the XPaths, search results are nicely displayed in the search results and
information panel. The name of the XPath is also chosen by the user. BitThief just uses the path value
entered by the user to get the information, and displays the name and information obtained in the
information panel.
Figure 8: BitThief’s Torrent Search Result Information Panel
12.
If the user has entered the query string and it fails to retrieve the information using the XPath, it simply
skips evaluation of entire query string and also the result.
OTHER FEATURES
As a part of this project, we also implemented a few other small, but useful features. Since a search
engine can have many fields filled in with complicated XPaths and regular expressions, we support
copying a search engine to the clipboard and adding a search engine from the clipboard. A similar
feature was implemented for the user profiles. These features blend in really well with the concept of
“power users”. Using this, smart users and friends can easily share their configurations on social media
platforms or forums and exchange it between friends.
13.
Results
With the design and features of BitThief’s new system, we were able to meet our goals of providing
flexibility and the support to the power users. Below, we explain how these goals were achieved.
FLEXIBILITY
From the figures of search engines and their components above, it is apparent that nearly each and every
field is usereditable. Whenever a good torrent website comes out, the user can simply add it. If any
torrent website is down or changes its layout, a user can simply remove or edit the URLs and XPaths.
Even the profile query strings are kept configurable. The profiles also automatically sync to search
engine changes. In this way, we meet our first goal of providing “flexibility”.
POWER USERS
BitThief, while it can act as a simple searchanddownload tool, it is apparent now that much amazing
stuff can be done with it. It does a very simple task: take input from user, retrieve and process the data
as user specified, and spit out the results. It does not care whether the data entered by the user is fully
correct or not. If a given XPath is wrong, it will simply not return any value. A smart user can further
use regular expressions to make the experience even better. For example, https://thepiratebay.org shows
a torrent upload date as 20150802 17:02:24 GMT. While XPath can retrieve this value, regular
expressions then allow the user to extract “2015” as year, “08” as month and “02” as day. Using this,
they can filter results using simple query strings like “Uploaded after ‘03 november 2014’”.
The behavior could further vary by how the user provides the URLs. For example, for torrent categories,
if the user doesn’t provide the URL with <index>, BitThief won’t automatically hop on to the next page
to find more results. Also, a smart user might provide BitThief with the modified link based on his need.
Instead of providing the default link https://thepiratebay.org/browse/201/<index>/3 for pirate bay movie
torrents, if the user is interested in movies having more seeders, they can rather use the “sort” feature on
the website and provide a modified link which is https://thepiratebay.org/browse/201/<index>/7.
We also have a debug dump file saved in the user’s home “bitthief” directory where they can easily
check out the retrieved page source, XPath information and query filter results. In this way, BitThief
also expects the user to be a debugger on their own in case the output isn’t as expected. It basically
expects its users to be “power users”.
14.
Conclusions
Starting with a simple userconfigurable metasearch engine, BitThief evolved into two different
versions in this project the current one and an intermediary IMDBintegration version.
Throughout the project many new things were learned. Before coming up with this UI, an intermediary
version of BitThief in this project provided only some limited flexibility.
Figure 9: BitThief’s Old Profile Manager showing noneditable fields
Users were able to carry out only two functions adding and deleting. None of the fields were kept
editable, which would turn out to be very frustrating as for one XPath change, users will have to recreate
an entire search engine entry. It was the same for the profiles as depicted by figure 9.
Also, the naming of components wasn’t clear to general users. For example, torrent categories were
called “lookups”. The dialog boxes had very complicated design with confusing layout and buttons. All
of these were pointed out by supervisors and corrected by the end of the term. “To keep the UI simple”
was something that I learned in this project.
FUTURE WORK
Even though the current version of BitThief is really robust with regards to flexibility, it still lacks
support for good debugging features. Currently, it dumps everything into a single file and it could be
painful and frustrating for a user to go over so much of the dumped data to find the information they
want. In future, maybe this could be enhanced for the debug files to contain indexed or mapped results
for each torrent search request, result or a link. If BitThief is unable to find data through XPath or unable
15.
to connect to the URL, a debug button could be displayed which directly fetches the indexed or mapped
data from the debug file, particularly for the current search, result, or a link. The UI can still be
improved a lot and made less confusing.
Now that we have a way to filter the torrent results easily, we can bring back the concept of integrating
different APIs like IMDB with the BitThief as plugins. A platform could be prepared for 3rd party
developers to develop for the BitThief where they can fetch a media list based on media category, rating
and other such filters and have a “search torrent” button which also asks user to enter the query string to
filter torrent results. In this way, users will be able to keep track of newly released or popular media,
search if a “good quality” torrent exists and download it, if required. If certain API goes down, user
should be simply able to get rid of that plugin. However, this would only be possible once the BitThief is
out there as a famous product and there are developers who are willing to make such small plugins.