This document discusses using web scraping techniques to collect bank offer data from websites. It describes how an offer scavenger software can automate the extraction of relevant data from websites and organize it in a predefined format like a database. The document then provides details on how the researchers collected bank offer data from websites like centralbank.net.in using web scraping and Python libraries. It explains the data extraction, transformation and loading process to clean the scraped data and load it into a database. Some preliminary statistics are also generated from the collected data. Finally, it discusses some legal aspects of using web scraping techniques.

1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 07 Issue: 04 | Apr 2020 www.irjet.net p-ISSN: 2395-0072
© 2020, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 166
Web Scraping Techniques to Collect Bank Offer Data from Bank Website
Jai Singh1, Dhrubojyoti Mookherjee2
1Student, Dept of Computer Science and Engineering, SRM Institute of Science and Technology, Chennai, India
2Student, Dept of Computer Science and Engineering, SRM Institute of Science and Technology, Chennai, India
----------------------------------------------------------------------***-------------------------------------------------------------------------
Abstract— Offer Scavenger software works in Application Programming Interface. It used to separate data from a website.
It is an Offer Scavenger uses web scratching and attains and documents required relevant data in a predestined arrangement.
It computerizes data separating from sites. The matter of a page is analyzed, examined, reformatted and manifested in a
database/spreadsheet or distributed storage. Web scratching, web collecting, or web information extraction is information
scratching utilized for removing information from sites. Web-scratching is utilized for contact-scratching, and as a segment of
utilizations utilized for web-ordering, web-mining and information mining, online value change checking and value
correlation, item survey scratching (to watch the challenge), accumulating land postings, climate information inspecting, site
change identification, look into, following on the web proximity and notoriety, web-mashup and, web information association.
Index Terms— Offer Scavenger, Web Scratching, Database, Web information, Web mashup, Correlation.
1. Introduction
Bank Offers have an important role, especially in the
economic field. By collecting Bank Offer it allows us to
analyze patterns of data from different banks. Some
research utilizing Bank Offer patterns are used to study
Bank Offer patterns for agricultural loans, in health
insurance, car loans, house loans. To estimate the Bank
Offer is difficult because the Bank Offer is dynamic.
Making Bank Offer estimates are profoundly relying on
perceived data and the methods of Bank Offer used. The
data helps to compute data and strengthen the analysis.
We need to observe data at some points to see the
different offers so that the Bank Offer is valid.
It is very difficult to receive the newest Bank Offer
data in a definite period in a delineated form due to the
method to every distinct firm. On the other side, few of
the sites like https://www.centralbank.net.in,
https://www.hdfcbank.com, https://www.onlinesbi.com,
produce renewed Bank Offer data online.
Here in this research, the data will be gathered from
those sites which provide real-time data utilizing web
scraping technology. The data is gathered from multiple
websites and then accumulates into one database or
spreadsheet using web scraping technology. It makes the
method more accessible to examine and anticipate the
accumulated data. A database or Bank Offer dataset will
form with the gathered data which is accumulated by
web scraping technique. This research is preliminary
research to prepare the Bank Offer dataset that will be
used for further research.
2. PREVIOUS WORK
Web scraping is a method of collecting data with the help
of a program which communicates with the Application
Programming Interface. The web scraping technique is
mostly done by building programs that automatically drive
queries to the web server, demanding data (normally in
HTML and distinct sorts of web pages), then to derive the
essential information the data is analyze. Web scraping
does multiple programming and technology methods, such
as data examination, general language analyzing and
protecting information.
Issues of usage the web scratching system itself is
generally examined in a few papers. One of the article [9]
in Pereira introduces the instruments and systems utilized
for scratching and its effect on informal organizations.
Other Utilization Techniques are utilized to gather rental
posting information from Craigslist site [11]. The
information gathered is utilized to investigate the lodging
market, human conduct and urban dynamic. A survey
conducted on a consumer price with web scraping
technique regarding consumer products related to
electronics (goods) and airfares by Polidoro et al. [12].
With the web scraping techniques saving time the outcome
show gathered statistical data.
Novkovic et al. [4] help to do the utilization of the web
scrapping which is associated with Bank Offer data. In
research, the traffic accident data gathered for 15 years
with the assistant of web scraping and connects it to
meteorological data. The outcomes present utilizing data
mining there are linkages of numerous Bank Offer
variables with the level of the traffic collision.

2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 07 Issue: 04 | Apr 2020 www.irjet.net p-ISSN: 2395-0072
© 2020, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 167
3. WEB SCRAPING BANK OFFER DATA
In the area of field study, assessment of consolidation of
Bank Offer data is done with web scratching. Field
research assessment does essential for examination inside
space of programming system structures, particularly for
programming system improvement. As for programming
system progression models that locale unit used abuse
dynamic models. This dynamic model parcels the bit of
programming structure progression step by step per it
perform. With this consistent improvement, there will be
changes in every movement of augmentations.
The system made during this examination is that the
hidden part or dynamic segment one. During this hidden
stage the event of a Bank supply information in India with
a web scratching methodology. Inside the accompanying
stage, learning accumulated inside the hidden part is sent
to the event of a Bank supply desire structure. At the last
step based on the Decision Support System, a Bank Offer
will be made. For database advancement, for example, the
phases of programming improvement as a rule through the
periods of investigation, structure, coding, and testing.
The process of web scraping Bank Offer data consists of
various steps as follows:
• The starting time frames inside the Analysis zone is
finding the structure of HTML reports from all destinations
which will be damaged. This procedure is done to type the
data and parts to be recuperated or keep.
• The accompanying stage is to make a dragging
undertaking made with Python substance using the
Beautiful Soup and Requests library. The outcomes of the
scratching data are exerted consider of in the outdo wants
to register.
• Make Task Scheduler run scripting data scratching
intermittently dependably. Auto task scheduler will
perform scratching data all through the site and extra it
into the results record.
• To isolate the crawling data the following method of
web scratching is used. system. Pentaho Kettle
instruments are used with the help of which extraction
method is performed. From the input obtained is
performed cleansing approach to manifest irrelevant
erudition, for instance, systems of the set away factors.
Change data to adjust courses of action and data
compositions as required (e.g., date and time
arrangement, city data, and anything are possible from
that point). Likewise, do the path toward merging files to
tie together the reports scratching into one record to
energize the assessment system. .
• Make bits of knowledge of Bank Offer data and
examination of data procured by the prerequisites of
utilization improvement.
4. RESULTS
4.1 Web Scraping Process Bank Offer Data
Running on Windows OS 10 in Anaconda platform using
HTML Analyzing Web Bank Offer Scraping Application
developed in Python programming language.
The first content will scrap the information on the site for
urban communities in India. The put away information
variable comprises a specific number of factors. The fickle
date is acquired via operating the date-time capacity
changed over to time-dependent upon the time zone
within Asia.
Fig.1: Architecture diagram
4.2 Data Extraction and Transformation
 Outcomes from the info files collected from the scraping
method cannot be undeviatingly used for the analytics
method because there are systems on the data fluctuating
attained. Also, it needs to fit the setup of the details read.
In Method data transmutation and uprooting, we utilize
tools Pentaho Data Integration.
 The ETL process which means Extract Transform Load
used for implementing the online Bank Offer data can be
viewed in Figure 1. This ETL process will extricate data
from scraping, modifying data by transforming content
into a structure that can be concocted. The data load
process is the ultimate process for uniting data and
manifesting data. In detail, the levels of the ETL method
are as follows:

3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 07 Issue: 04 | Apr 2020 www.irjet.net p-ISSN: 2395-0072
© 2020, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 168
 The Conversion strategy is arranged by making an
information change archive for 8 urban territories. The
change method of the individual city will examine the
culminating record of the uprooting information
concerning every city as prototype Primulin report. The
result of the rudimentary progress is to scrutinize the csv
archive including a while later modify the data
assemblage as needed. The accompanying stage
methodology is to arrange data cleansing (exclude the
unit from the information variable). Endured progress
Strings Cut Rain that converts turbulent area information
to incorporates Kota will consolidate the depiction of city
information and extension (if essential).
 Likewise, the eventual outcome will be stored in the Bank
Offer document.
 In place of doing concurrently, the process is done
consecutively so that equivocation of the data which is
caused by file access can be circumvented.
 To drive the complete change method, a work that will
run the whole city change process in progression. The
system is done successively rather than concurrently to
circumvent data frictions obtained by record approach to
concurrently.
 Bank Offer sumsel.csv record which is the resulting
archive for the eight viewed urban networks arranged to
utilize in the evaluation data method. This action
technique itself can be gotten together by the action to
load the records scratching to the provincial PC on an
arranged reason. Occupation technique can drive
normally with assignment organizer. Example- the plan of
including data for assessment performed each day at
precise man-hours. The action directions seek following
modified the path toward isolating the download is
completed.
 The ETL procedure for the information scratching results
from Bank Offer .com and time and date. com results are
like the earlier technique. Be that as it may, the procedure
is progressively clear. In the two documents are just done
cleaning procedure, remove and change without blending
the information.
4.3 Data Statistic and Analytic Process
Statistics are the form of a result of the accumulated
information which is prepared by web scraping methods.
Data statistics are generated utilizing Python
programming language.
The examination itself is another fundamental
investigation to the phase of gathering Bank Offer
information in India. In this manner, the procedure of
investigation information for gathered Bank Offer
information is as yet restricted to the introduction of
factual data. While the investigative technique in detail for
instance for Bank Offer forecast, Bank Offer example
mapping is impossible. This is because the Bank Offer
information-gathering phase that drives just driven
individual month. In this way, information gathered is yet
moderately little which is around 5909 records. The
information can't be utilized to produce expectations and
Bank Offer gauges. Since the forecast procedure itself in a
perfect world uses a ton of Bank Offer information to
create precise appraisals of great outcomes. For the later
information accumulation method with the network,
scratching procedures will keep on being done
persistently. After the information gathered enough (over
one year), at that point, the following phase of research
will be made.
Different types of true data in this investigation are
additionally introduced in the method for outlines and
designs. With the introduction as graph function it simpler
for clients to see the measurements of Bank Offer
information from every city to be watched. Likewise, the
histogram can be introduced in every day, month to month
and annually structures as needed by utilizing the
gathering capacity by the Date variable.
Bank Offer expectation utilizing the methodology of
information mining and AI. Additionally, the future after
effects of information gathered by web scratching systems
will likewise be utilized to ponder the importance of Bank
Offer examples for basic leadership in the area of transport
and horticulture.
4.4 Legal Aspect Issues
The lawfulness and reasonable utilization of the utilization
of web scratching methods is frequently an issue. There
are two perspectives to consider in doing web scratching
strategies, to be specific copyright and section without
consent [19].
• First, for copyright concerns, a government field
court on operating on scarping information isn't a
copyright encroachment for freely accessible
information. In the investigation of the utilization
of web scratching systems in this Bank Offer
information, the scratching site presents
information freely. Additionally, this exploration
accesses all the time doesn't trouble the webpage
(just one solicitation for each hour) and doesn't
harm the site hosts of the got to information.
• For the subsequent viewpoint, enter without
consent. During the time spent scratching web
that we made the site open openly available
unreservedly. IP utilized at the hour of the
exploration is certainly not a blocked IP, and

4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 07 Issue: 04 | Apr 2020 www.irjet.net p-ISSN: 2395-0072
© 2020, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 169
access is managed without bypassing any
intermediary and the information got to isn't
encoded information. In this way, from the part of
the legitimateness of the web scratching
procedure is done doesn't abuse anything. The
aftereffects of information gathered are not
utilized for business purposes yet research
purposes. The examination procedure likewise
doesn't repack or rehash the information yet to
be investigated.
5. Conclusion
In this research work, to accumulate bank offers web
scrapping has been successfully used . In this process, we
actually gather multiple data from various websites like
Bank Offer and information about shares etc. The user who
is looking for bank offers can directly use that offers which
we store in the storage system in cloud, Database
Management System. They don't have to visit particular
bank websites to view offers. Each vendor has to put his
manpower to extract and note this information and use it
in their preferred way. Offer Scavenger is a solution which
automates the data extracting and providing a service to
make this data available to cloud storage and can be used
by vendors.
6. ACKNOWLEDGEMENT
We express special gratitude towards Ms Caroline who has
guided this research.
7. REFERENCES
1] C. Lesk, P. Rowhani, and N. Ramankutty, "Impact of
extraordinary climate fiascos on worldwide yield
creation," Nature, vol. 529, no. 7584, pp. 84–87, Jan. 2016
[2] J. H. Hashi[m and Z. Hashim, "Environmental change,
extraordinary climate occasions, and human wellbeing
suggestions in the Asia Pacific locale," Asia Pacific Journal
of Public Health, vol. 28, no. 2_suppl, pp. 8S–14S, 2016.
[3] G. J. Zheng et al., "Investigating the serious winter fog in
Beijing: the effect of concise climate, provincial vehicle and
heterogeneous responses," Atmospheric Chemistry and
Physics, vol. 15, no. 6, pp. 2969–2983, Mar. 2015.
[4] M. Novkovic, M. Arsenic, S. Blagojevich, A. Anderle, and
D. Stefanovic, “Data science applied to extract insights
from data - weather data influence on traffic accidents,” p.
7.
[5] M. Hebert, “Climatology for city planning in historical
perspective,” Urban Climate, vol. 10, pp. 204–215, Dec.
2014.
[6] R. Mitchell, Web Scraping with Python: Collecting More
Data from the Modern Web. O’Reilly Media, Inc., 2018.
[7] J. I. Fernández Villamor, J. Belasco Garcia, C. A. Iglesias
Fernandez, and M. GarijoAyestaran, “A semantic scraping
model for web resources Applying linked data to web page
screen scraping,” 2011.