This series of slides describes how to develop a twitter application.
This slide shows how to search tweets using Twitter Search RESTful Open API and how to implement it using Twitter4J.
Secure Enterprise APIs for Mobile, Cloud & Open Web
APIs present enterprises with many business opportunities but they also create new attack vectors that hackers can potentially exploit. APIs share many of the same threats that plague the Web but APIs are fundamentally different from Web sites and have an entirely unique risk profile that must be addressed.
By adopting a secure API architecture from the beginning, it is possible to address both old and new threats. In this webinar, Scott Morrison – CTO at Layer 7 Technologies – will explain in detail how an enterprise can pursue its API publishing strategy without compromising the security of its on-premise systems and data.
You Will Learn
How APIs increase the attack surface
What key types of risk are introduced by APIs
How enterprises can mitigate each of these risks
Why it is crucial to separate API implementation and security into distinct tiers
Presented By
Scott Morrison, CTO, Layer 7 Technologies
Secure Enterprise APIs for Mobile, Cloud & Open Web
APIs present enterprises with many business opportunities but they also create new attack vectors that hackers can potentially exploit. APIs share many of the same threats that plague the Web but APIs are fundamentally different from Web sites and have an entirely unique risk profile that must be addressed.
By adopting a secure API architecture from the beginning, it is possible to address both old and new threats. In this webinar, Scott Morrison – CTO at Layer 7 Technologies – will explain in detail how an enterprise can pursue its API publishing strategy without compromising the security of its on-premise systems and data.
You Will Learn
How APIs increase the attack surface
What key types of risk are introduced by APIs
How enterprises can mitigate each of these risks
Why it is crucial to separate API implementation and security into distinct tiers
Presented By
Scott Morrison, CTO, Layer 7 Technologies
This webinar in the course of the LOD2 webinar series will present the release 2.0 of the LOD2 stack, which contains updates to the components Ontowiki, Silk
* the assisting sparql editor SPARQLED (DERI),
* the LOD enabled Open Refine (previously Google Refine) (ZEMANTA),
* the extended version of SILK with link suggestion management from LATC (DERI),
* the rdfAuthor library which allows to manage structured information from RDFa-enhanced websites (ULEI),
* the SPARQLPROXY which is a PHP based forward proxy for remote access to SPARQL end points (ULEI)
Release 2.0 contains also a first contributed debian package for a component which is maintained by a group outside the LOD2 consortium. With the help of ULEI a package for the STANBOL engine (http://stanbol.apache.org/) has been contributed.
If you are interested in Linked (Open) Data principles and mechanisms, LOD tools & services and concrete use cases that can be realised using LOD then join us in the free LOD2 webinar series!
http://lod2.eu/BlogPost/webinar-series
This webinar in the course of the LOD2 webinar series will present Zemanta and its LODRefine - a LOD-enabled version of OpenRefine (previously Google Refine), which is a part of the LOD2 stack. LODRefine extends cleansing and linking functionalities of OpenRefine by providing means to reconcile and augment your data with DBpedia or any other SPARQL endpoint, extract named entities using Zemanta API, export data in one of the RDF formats, and recently also to exploit available crowdsourcing services. In webinar we will demonstrate several task which demonstrate the ease of use and versatility of LODRefine.
If you are interested in Linked (Open) Data principles and mechanisms, LOD tools & services and concrete use cases that can be realised using LOD then join us in the free LOD2 webinar series: http://lod2.eu/BlogPost/webinar-series
Intro to Twitter & social media concepts for use in CME. Follow us at http://twitter.com/EinsteinMed. Related video interviews at end of presentation and at http://bit.ly/OW5wiT and http://bit.ly/TWuZXP
Welcome to my series of articles on Unified Modeling Language. This is "Session 10 – Sequence Diagram" of the series. Please view my other documents where I have covered each UML diagram with examples
REST Assured is a Java library .Which, provides a domain-specific language (DSL) for writing powerful, maintainable tests for Restful APIs.
Library behaves like a headless Client to access REST web services.
This webinar in the course of the LOD2 webinar series will present the release 2.0 of the LOD2 stack, which contains updates to the components Ontowiki, Silk
* the assisting sparql editor SPARQLED (DERI),
* the LOD enabled Open Refine (previously Google Refine) (ZEMANTA),
* the extended version of SILK with link suggestion management from LATC (DERI),
* the rdfAuthor library which allows to manage structured information from RDFa-enhanced websites (ULEI),
* the SPARQLPROXY which is a PHP based forward proxy for remote access to SPARQL end points (ULEI)
Release 2.0 contains also a first contributed debian package for a component which is maintained by a group outside the LOD2 consortium. With the help of ULEI a package for the STANBOL engine (http://stanbol.apache.org/) has been contributed.
If you are interested in Linked (Open) Data principles and mechanisms, LOD tools & services and concrete use cases that can be realised using LOD then join us in the free LOD2 webinar series!
http://lod2.eu/BlogPost/webinar-series
This webinar in the course of the LOD2 webinar series will present Zemanta and its LODRefine - a LOD-enabled version of OpenRefine (previously Google Refine), which is a part of the LOD2 stack. LODRefine extends cleansing and linking functionalities of OpenRefine by providing means to reconcile and augment your data with DBpedia or any other SPARQL endpoint, extract named entities using Zemanta API, export data in one of the RDF formats, and recently also to exploit available crowdsourcing services. In webinar we will demonstrate several task which demonstrate the ease of use and versatility of LODRefine.
If you are interested in Linked (Open) Data principles and mechanisms, LOD tools & services and concrete use cases that can be realised using LOD then join us in the free LOD2 webinar series: http://lod2.eu/BlogPost/webinar-series
Intro to Twitter & social media concepts for use in CME. Follow us at http://twitter.com/EinsteinMed. Related video interviews at end of presentation and at http://bit.ly/OW5wiT and http://bit.ly/TWuZXP
Welcome to my series of articles on Unified Modeling Language. This is "Session 10 – Sequence Diagram" of the series. Please view my other documents where I have covered each UML diagram with examples
REST Assured is a Java library .Which, provides a domain-specific language (DSL) for writing powerful, maintainable tests for Restful APIs.
Library behaves like a headless Client to access REST web services.
Covers topics like RestApi, Authentication, Authorization, Cacheability, Advantage of RestApi over SOAP and some basics regarding Open Data Protocol(OData).
THE FOLLOWING SUN CERTIFIED WEBCOMPONENT DEVELOPER FOR J2EEPLATFORM EXAM OBJECTIVES COVERED IN THIS CHAPTER:
1.1 For each of the HTTP methods,GET,POST, and PUT, identify the corresponding method in the HttpServlet class.
1.2 For each of the HTTP methods,GET ,POST, and HEAD, identify triggers that might cause a browser to use the method, and identify benefits or functionality of the method.
Introduce the Java Enterprise (J2EE) model
Present the Hypertext Markup Language (HTML) tags
Present the Hypertext Transmission Protocol (HTTP)
Define an HTTP client request, server response, and HTTP request methods
JSP 프로그래밍 #02 서블릿과 JSP 시작하기
2.1 톰캣 애플리케이션 만들기 (온라인 강의: https://youtu.be/04LIGWKCFjY)
2.2 간단한 서블릿 만들기 (온라인 강의: https://youtu.be/4ajw5EsxYE8)
2.3 간단한 JSP 만들기 (온라인 강의: https://youtu.be/6h-qH8pGdT8)
2.4 간단한 자바빈즈 만들기 (온라인 강의: https://youtu.be/TlgXkAWi1sc)
JSP 프로그래밍 #01 웹 프로그래밍
1.1 웹 (온라인 강의: https://youtu.be/qDZXXHhMr4A)
1.2 서블릿 (온라인 강의: https://youtu.be/a8hHeUhbz2k)
1.3 JSP(Java Server Page) (온라인 강의: https://youtu.be/Q4ezLP6KLwM)
1.4 프로그래밍을 위한 환경 설정 (온라인 강의: https://youtu.be/k2eR6gLULA8)
2018년 7월 5일에 있었던 한국인터넷거버넌스포럼(KrIGF)에서 발표한 "오픈 데이터와 인공지능" 발표자료입니다.
다음과 같은 내용을 담고 있습니다.
* 오픈데이터의 정의
* 오픈데이터의 중요성
* 인공지능
* 인공지능에서 데이터의 중요성
* 제한된 데이터 환경에서의 문제점
* 인공지능을 위한 오픈데이터의 중요성
* 더 나은 인공지능 시대를 위한 제언
2017년 4월에 진행된 도서관최신동향 과정에 있었던 발표자료입니다.
서지 분야에서의 Linked Data의 개념과 활용에 대한 내용을 담고 있습니다.
구체적으로는 아래와 같은 내용을 포함합니다.
- Linked Data란 무엇인가?
- 왜 도서관에서 Linked Data를 이야기하는가?
- Linked Data를 누가 쓰고 있나?
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Assure Contact Center Experiences for Your Customers With ThousandEyes
Development of Twitter Application #8 - Streaming API
1. Linked Data &
Semantic Web
Technology
Development of
Twitter Applications
Part 8. Streaming API
Dr. Myungjin Lee
2. Linked Data & Semantic Web Technology
Streaming API
• Overview
– low latency access to Twitter's global stream of Tweet
data
– A proper implementation of a streaming client will be
pushed messages indicating Tweets and other events
have occurred.
• Streaming Endpoints
2
Public streams
Streams of the public data flowing through Twitter.
Suitable for following specific users or topics, and data
mining.
User streams
Single-user streams, containing roughly all of the data
corresponding with a single user's view of Twitter.
Site streams
The multi-user version of user streams. Site streams are
intended for servers which must connect to Twitter on
behalf of many users.
3. Linked Data & Semantic Web Technology
Differences between Streaming and REST
• REST API
• Streaming API
3
4. Linked Data & Semantic Web Technology
Connecting to a Streaming Endpoint
• Authentication
• Connecting
– form a HTTP request and consume the resulting
stream
– holding the connection open indefinitely, barring
server-side error, excessive client-side lag, network
hiccups, routine server maintenance or duplicate
logins
• Disconnections
– too many connections with the same credentials
– to stop reading data
– reading data too slowly
– a streaming server is restarted
– twitter’s network configuration changes
4
Auth Type Supported APIs Description
Basic Public Streams
Requests include the credentials of a
valid Twitter account.
OAuth
Public Streams
User Streams
Site Streams
Requests must be authorized according to
the OAuth specification.
5. Linked Data & Semantic Web Technology
Connecting to a Streaming Endpoint
• Stalls
– If 90 seconds pass with no data received, including
newlines, disconnect and reconnect immediately.
– The Streaming API will send a keep-alive newline
every 30 seconds to prevent your application from
timing out the connection.
• Reconnecting
– reconnecting attempts according to the type of error
experienced
• TCP/IP level network errors
– reconnect by 250ms each attempt, up to 16 seconds
• HTTP errors
– start with a 5 second wait, doubling each attempt, up to
320 seconds
• HTTP 420 errors
– start with a 1 minute wait and double each attempt
5
6. Linked Data & Semantic Web Technology
Connecting to a Streaming Endpoint
• Connection Churn
– churn: repeatedly opening and closing a connection
– keep your connections as stable and long-lived as
possible
• Rate Limiting
– Client which do not implement backoff and attempt to
reconnect as often as possible will have their
connections rate limited for a small number of
minutes.
– Rate limited clients will receive HTTP 420 responses
for all connection requests.
– If your client is rate limited frequently, it is possible
that your IP will be blocked from accessing Twitter for
an indeterminate period of time.
6
7. Linked Data & Semantic Web Technology
Connecting to a Streaming Endpoint
• HTTP Error Codes
7
Status Text Description
200 Success Self evident.
401 Unauthorized
HTTP authentication failed due to either:
•Invalid basic auth credentials, or an invalid OAuth request.
•Out-of-sync timestamp in your OAuth request (the response body will indicate
this).
•Too many incorrect passwords entered or other login rate limiting.
403 Forbidden The connecting account is not permitted to access this endpoint.
404 Unknown There is nothing at this URL, which means the resource does not exist.
406
Not
Acceptable
At least one request parameter is invalid. For example, the filter endpoint returns
this status if:
•The track keyword is too long or too short.
•An invalid bounding box is specified.
•Neither the track nor follow parameter are specified.
•The follow user ID is not valid.
413 Too Long
A parameter list is too long. For example, the filter endpoint returns this status if:
•More track values are sent than the user is allowed to use.
•More bounding boxes are sent than the user is allowed to use.
•More follow user IDs are sent than the user is allowed to follow.
416
Range
Unacceptable
For example, an endpoint returns this status if:
•A count parameter is specified but the user does not have access to use the count
parameter.
•A count parameter is specified which is outside of the maximum/minimum
allowable values.
420 Rate Limited
The client has connected too frequently. For example, an endpoint returns this status
if:
•A client makes too many login attempts in a short period of time.
•Too many copies of an application attempt to authenticate with the same
credentials.
503
Service
Unavailable
A streaming server is temporarily overloaded. Attempt to make another connection,
keeping in mind the connection attempt rate limiting and possible DNS caching in
your client.
8. Linked Data & Semantic Web Technology
Streaming API Request Parameters
• delimited
– be used on all streaming endpoints
– delimited=length
• statuses should be delimited in the stream by a length,
in bytes, a newline, and the status text
8
9. Linked Data & Semantic Web Technology
Streaming API Request Parameters
• stall_warnings
– be used on all streaming endpoints
– stall_warnings=true
• periodic messages to be delivered every 5 minutes if the
client is in danger of being disconnected
• filter_level
– be used on all streaming endpoints
– to set the minimum value of the Tweet attribute
required to be included in the stream
– one of none, low, and medium
– the default value is none.
• language
– be used on all streaming endpoints
– to return Tweets that have been detected as being
written in the specified languages
9
10. Linked Data & Semantic Web Technology
Streaming API Request Parameters
• follow
– available on POST statuses/filter
– a comma-separated list of user IDs
– to indicate the users whose Tweets should be
delivered on the stream
– the stream contain:
• Tweets created by the user.
• Tweets which are retweeted by the user.
• Replies to any Tweet created by the user.
• Retweets of any Tweet created by the user.
• Manual replies, created without pressing a reply button
(e.g. “@twitterapi I agree”).
– the stream does not contain:
• Tweets mentioning the user (e.g. “Hello @twitterapi!”).
• Manual Retweets created without pressing a Retweet
button (e.g. “RT @twitterapi The API is great”).
• Tweets by protected users.
10
11. Linked Data & Semantic Web Technology
Streaming API Request Parameters
• track
– available on POST statuses/filter
– what Tweets will be delivered on the stream
• a comma-separated list of phrases that may be one or more terms
separated by spaces
• a phrase will match if all of the terms in the phrase are present in
the Tweet, regardless of order and ignoring case
• commas as logical ORs and white spaces as logical ANDs
• each phrase must be between 1 and 60 bytes
• exact matching of phrases is not supported
• punctuation and special characters will be considered part of the
term they are adjacent to
– to check the text attribute of the Tweet, expanded_url and
display_url for links and media, text for hashtags, and
screen_name for user mentions
11
12. Linked Data & Semantic Web Technology
Streaming API Request Parameters
• locations
– available on POST statuses/filter
– to specify a set of bounding boxes to filter Tweets
– a comma-separated list of longitude, latitude pairs
– conditions
• •If the coordinates field is populated, the values there will
be tested against the bounding box.
• •If coordinates is empty but place is populated, the region
defined in place is checked .
• If none of the rules listed above match, the Tweet does not
match the location query.
• count
– to attempt to backfill missed messages which occurred
during the disconnect periot
– an integer from 1 to 150000 or from -1 to -150000
– use of this parameter should be carefully considered, as
high values increase the chance of a subsequent
disconnect
12
13. Linked Data & Semantic Web Technology
Streaming API Request Parameters
• with
– available on GET users and GET site
– to controls the types of messages delivered to User
and Site Streams clients
– the default is with=user
– The value with=followings adds messages from
accounts the user follows, equivalent to the user's
home timeline.
• replies
– available on GET users and GET site
– replies=all
• to have mimics of all Tweets returned in a streaming
connection
13
14. Linked Data & Semantic Web Technology
Processing Streaming Data
• Parsing responses
– The body of a streaming API response consists of a
series of newline-delimited messages.
• "newline" is considered to be rn (in hex, 0x0D 0x0A)
• "message" is a JSON encoded data structure or a blank
line
• JSON data
– The individual messages streamed by this API are
JSON encoded.
– the attributes of a JSON-encoded object are unordered
• Missing counts
– In very rare cases, the Streaming API may elect to
deliver an incomplete Tweet field.
14
15. Linked Data & Semantic Web Technology
Processing Streaming Data
• Transfer-Encoding: chunked
– Most streaming connections will be encoded using
chunked transfer encoding, as a Transfer-
Encoding: chunked HTTP header in the
response.
• Delimited messages
– Each message will be preceded by a string
representation of a base-10 integer indicating the
length of the message in bytes.
• Falling behind
– A way to track whether your client is falling behind is
to compare the timestamp of the Tweets you receive
with the current time.
– If the difference between the timestamps increases
over time, then the client is not processing Tweets as
fast as they are being delivered.
15
16. Linked Data & Semantic Web Technology
Processing Streaming Data
• Message ordering
– Messages from the Streaming API are not delivered in
sorted order.
– Delete messages may be delivered before the original
Tweet so implementations should be able to replay a
local cache of unrecognized deletes.
• Duplicate messages
– Duplicate messages may be delivered so
implementations should be tolerant of receiving a
Tweet more than once.
• Gzip compression
– Gzip compression may reduce the bandwidth needed
to process a stream to as small as 1/5th the size of an
uncompressed stream.
16
17. Linked Data & Semantic Web Technology
Public Streams
• Overview
– samples of the public data flowing through Twitter
• Endpoints
– GET statuses/sample
– GET statuses/firehose
– POST statuses/filter
• Using the streaming APIs
– connect to the endpoinsts
– process data returned by the streaming APIs
– There are some streaming messages which are not
returned by the REST APIs. See streaming message
types for information.
• Connections
– each account may create only one standing connection
to the public endpoints
– connecting to a public stream more than once with the
same account credentials will cause the oldest
connection to be disconnected
17
18. Linked Data & Semantic Web Technology
Public Stream Messages Types
• Blank lines
– some messages may be blank lines which serve as
"keep-alive" signals to prevent clients have stalled and
closing the connection.
• Status deletion notices (delete)
– These messages indicate that a given Tweet has been
deleted.
• Location deletion notices (scrub_geo)
– to indicate that geolocated data must be stripped from
a range of Tweets
18
19. Linked Data & Semantic Web Technology
Public Stream Messages Types
• Limit notices (limit)
– to indicate that a filtered stream has matched more
Tweets than its current rate limit allows to be
delivered
– Limit notices contain a total count of the number of
undelivered Tweets.
• Stall warnings (warning)
– When connected to a stream using the stall_warnings
parameter, you may receive status notices indicating
the current health of the connection.
19
20. Linked Data & Semantic Web Technology
Public Stream Messages Types
• Disconnect messages (disconnect)
– to deliver a message indicating why a stream was
closed
– status codes
20
Code Name Description
1 Shutdown The feed was shutdown (possibly a machine restart)
2
Duplicate
stream
The same endpoint was connected too many times.
3 Control request Control streams was used to close a stream (applies to sitestreams).
4 Stall The client was reading too slowly and was disconnected by the server.
5 Normal The client appeared to have initiated a disconnect.
6 Token revoked An oauth token was revoked for a user (applies to site and userstreams).
7 Admin logout
The same credentials were used to connect a new stream and the oldest was
disconnected.
8 Reserved for internal use. Will not be delivered to external clients.
9
Max message
limit
The stream connected with a negative count parameter and was disconnected
after all backfill was delivered.
10
Stream
exception
An internal issue disconnected the stream.
11 Broker stall An internal issue disconnected the stream.
12 Shed load
The host the stream was connected to became overloaded and streams were
disconnected to balance load. Reconnect as usual.
21. Linked Data & Semantic Web Technology
GET statuses/sample
• sample Streaming API
– Returns a small random sample of all public statuses.
The Tweets returned by the default access level are the
same, so if two different clients connect to this
endpoint, they will see the same Tweets.
• Resource URL
– https://stream.twitter.com/1.1/statuses/sample.json
• Parameters
• Other Information
– Rate Limited? Yes
– Authentication: Requires user context
– Response Object: Tweets
21
delimited
optional
Specifies whether messages should be length-delimited.
stall_warnings
optional
Specifies whether stall warnings should be delivered.
22. Linked Data & Semantic Web Technology
Twitter4J Classes for sample Streaming
• TwitterStreamFactory Class
– A factory class for TwitterFactory.
– An instance of this class is completely thread safe and
can be re-used and used concurrently.
– Methods
• TwitterStream getInstance()
– Returns a instance associated with the configuration
bound to this factory.
• TwitterStream getInstance(AccessToken accessToken)
– Returns a OAuth Authenticated instance.
– consumer key and consumer Secret must be provided by
twitter4j.properties, or system properties.
• TwitterStream Interface
– Methods
• void addListener(StatusListener listener)
• void sample()
– Starts listening on random sample of all public statuses.
• void shutdown()
– Shuts down internal dispatcher thread shared by all
TwitterStream instances.
22
23. Linked Data & Semantic Web Technology
Twitter4J Classes for sample Streaming
• StatusListener Interface
– Methods
• void onDeletionNotice(StatusDeletionNotice statusDeletionNotice)
– Called upon deletionNotice notices.
• void onScrubGeo(long userId, long upToStatusId)
– Called upon location deletion messages.
• void onStallWarning(StallWarning warning)
– Called when receiving stall warnings.
• void onStatus(Status status)
• void onTrackLimitationNotice(int numberOfLimitedStatuses)
– This notice will be sent each time a limited stream becomes
unlimited.
23
24. Linked Data & Semantic Web Technology
Example of sample Streaming API
1. import twitter4j.StallWarning;
2. import twitter4j.Status;
3. import twitter4j.StatusDeletionNotice;
4. import twitter4j.StatusListener;
5. public class StatusListenerImpl implements StatusListener {
6. @Override
7. public void onStatus(Status status) {
8. System.out.println("@" + status.getUser().getScreenName() + " - "
9. + status.getText());
10. }
11. @Override
12. public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
13. System.out.println("Got a status deletion notice id:"
14. + statusDeletionNotice.getStatusId());
15. }
16. @Override
17. public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
18. System.out.println("Got track limitation notice:"
19. + numberOfLimitedStatuses);
20. }
21. @Override
22. public void onScrubGeo(long userId, long upToStatusId) {
23. System.out.println("Got scrub_geo event userId:" + userId
24. + " upToStatusId:" + upToStatusId);
25. }
26. @Override
27. public void onStallWarning(StallWarning warning) {
28. System.out.println("Got stall warning:" + warning);
29. }
30. @Override
31. public void onException(Exception ex) {
32. ex.printStackTrace();
33. }
34. }
24
25. Linked Data & Semantic Web Technology
Example of sample Streaming API
1. import twitter4j.*;
2. public class PrintSampleStream {
3. public static void main(String[] args) throws TwitterException {
4. TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
5. twitterStream.setOAuthConsumer(TwitterAccessToken.consumerKey,
6. TwitterAccessToken.consumerSecret);
7. twitterStream.setOAuthAccessToken(TwitterAccessToken.loadAccessToken());
8. StatusListenerImpl listener = new StatusListenerImpl();
9. twitterStream.addListener(listener);
10. twitterStream.sample();
11. }
12. }
25
26. Linked Data & Semantic Web Technology
GET statuses/firehose
• firehose Streaming API
– This endpoint requires special permission to access.
– Returns all public statuses.
– Few applications require this level of access.
– Creative use of a combination of other resources and
various access levels can satisfy nearly every
application use case.
• Resource URL
– https://stream.twitter.com/1.1/statuses/firehose.json
• Parameters
• Other Information
– Rate Limited? Yes
– Authentication: Requires user context
– Response Object: Tweets
26
count
optional
The number of messages to backfill.
delimited
optional
Specifies whether messages should be length-delimited.
stall_warnings
optional
Specifies whether stall warnings should be delivered.
27. Linked Data & Semantic Web Technology
Twitter4J Classes for firehose Streaming
• Twitter4J Classes for firehose Streaming
– TwitterStream Interface
• Methods
– void firehose(int count)
» Starts listening on all public statuses.
• Example of firehose Streaming API
1. import twitter4j.*;
2. public class PrintFirehoseStream {
3. public static void main(String[] args) throws TwitterException {
4. TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
5. twitterStream.setOAuthConsumer(TwitterAccessToken.consumerKey,
6. TwitterAccessToken.consumerSecret);
7. twitterStream.setOAuthAccessToken(TwitterAccessToken.loadAccessToken());
8. StatusListenerImpl listener = new StatusListenerImpl();
9. twitterStream.addListener(listener);
10. twitterStream.firehose(0);
11. }
12. }
27
28. Linked Data & Semantic Web Technology
POST statuses/filter
• filter Streaming API
– Returns public statuses that match one or more filter
predicates.
– Both GET and POST requests are supported, but GET
requests with too many parameters may cause the request
to be rejected for excessive URL length.
• Resource URL
– https://stream.twitter.com/1.1/statuses/filter.json
• Parameters
• Other Information
– Rate Limited? Yes
– Authentication: Requires user context
– Response Object: Tweets
28
follow
A comma separated list of user IDs, indicating the users to return statuses for in the
stream.
track Keywords to track. Phrases of keywords are specified by a comma-separated list.
locations Specifies a set of bounding boxes to track.
delimited
optional
Specifies whether messages should be length-delimited.
stall_warnings
optional
Specifies whether stall warnings should be delivered.
29. Linked Data & Semantic Web Technology
Twitter4J Classes for filter Streaming
• TwitterStream Interface
– Methods
• void filter(FilterQuery query)
– Start consuming public statuses that match one or more
filter predicates.
• FilterQuery Class
– Methods
• FilterQuery count(int count)
– Sets count
• FilterQuery follow(long[] follow)
– Sets follow
• FilterQuery locations(double[][] locations)
– Sets locations
• FilterQuery track(java.lang.String[] track)
– Sets track
29
30. Linked Data & Semantic Web Technology
Example of filter Streaming API
1. import twitter4j.*;
2. public final class PrintFilterStream {
3. public static void main(String[] args) throws TwitterException {
4. StatusListenerImpl listener = new StatusListenerImpl();
5. TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
6. twitterStream.setOAuthConsumer(TwitterAccessToken.consumerKey,
7. TwitterAccessToken.consumerSecret);
8. twitterStream.setOAuthAccessToken(TwitterAccessToken.loadAccessToken());
9. twitterStream.addListener(listener);
10. FilterQuery fq = new FilterQuery();
11. // long[] follow = { 22931893, 16227629, 464020027, 299014098, 40203602,
12. // 121858259, 631481692, 173523502, 64972381, 357617589, 77689885 };
13. // fq.follow(follow);
14. // String[] track = { "psy", "#psy", "@psy_oppa", "gentleman",
15. // "#gentleman", "@ygent_official" };
16. // fq.track(track);
17. // double[][] locations = { { -74, 40 }, { -73, 41 } }; // New York City
18. // fq.locations(locations);
19. twitterStream.filter(fq);
20. }
21. }
30
31. Linked Data & Semantic Web Technology
User Streams
• Overview
– a stream of data and events specific to the
authenticated user
• Endpoint
– GET user
• Connections
– limited to only a few simultaneous User Streams
connections per OAuth application, regardless of IP
– to handle the HTTP 420 error code that indicates that
the account has been logging in to often
• User streams messages
– Types of messages
• some streaming messages which are not returned by the
REST APIs
31
32. Linked Data & Semantic Web Technology
User Stream Messages Types
• Friends lists (Friends)
– a preamble which is a list of the user’s friends before
starting regular message delivery
• Too many follows (warning)
– warning message when your application connects on
behalf of a user who follows more than 10,000
accounts
32
33. Linked Data & Semantic Web Technology
User Stream Messages Types
• Events (event)
– notifications about non-Tweet events
33
Description Event Name Source Target
Target
Object
User blocks someone block Current user Blocked user Null
User removes a block unblock Current user
Unblocked
user
Null
User favorites a Tweet favorite Current user Tweet author Tweet
User's Tweet is favorited favorite Favoriting user Current user Tweet
User unfavorites a Tweet unfavorite Current user Tweet author Tweet
User's Tweet is unfavorited unfavorite
Unfavoriting
user
Current user Tweet
User follows someone follow Current user Followed user Null
User is followed follow Following user Current user Null
User unfollows someone unfollow Current user Followed user Null
User creates a list list_created Current user Current user List
User deletes a list list_destroyed Current user Current user List
User edits a list list_updated Current user Current user List
User adds someone to a list list_member_added Current user Added user List
User is added to a list list_member_added Adding user Current user List
User removes someone
from a list
list_member_removed Current user Removed user List
User is removed from a list list_member_removed Removing user Current user List
User subscribes to a list list_user_subscribed Current user List owner List
User's list is subscribed to list_user_subscribed
Subscribing
user
Current user List
User unsubscribes from a
list
list_user_unsubscribed Current user List owner List
User's list is unsubscribed
from
list_user_unsubscribed
Unsubscribing
user
Current user List
User updates their profile user_update Current user Current user Null
34. Linked Data & Semantic Web Technology
GET user
• user Streaming API
– streams messages for a single user, as described in
User streams
• Resource URL
– https://userstream.twitter.com/1.1/user.json
• Parameters
• Other Information
– Rate Limited? Yes
– Authentication: Requires user context
34
delimited
optional
Specifies whether messages should be length-delimited.
stall_warnings
optional
Specifies whether stall warnings should be delivered.
with
optional
Specifies whether to return information for just the authenticating user, or include
messages from accounts the user follows.
replies
optional
Specifies whether to return additional @replies.
track
optional
Includes additional Tweets matching the specified keywords. Phrases of keywords
are specified by a comma-separated list.
locations
optional
Includes additional Tweets falling within the specified bounding boxes.
35. Linked Data & Semantic Web Technology
Twitter4J Classes for user Streaming
• TwitterStream Interface
– Methods
• void user()
– User Streams provides real-time updates of all data needed to update a
desktop application display.
• void user(java.lang.String[] track)
– User Streams provides real-time updates of all data needed to update a
desktop application display.
• UserStreamListener Interface
– Methods
• void onBlock(User source, User blockedUser)
• void onDeletionNotice(long directMessageId, long userId)
• void onDirectMessage(DirectMessage directMessage)
• void onFavorite(User source, User target, Status favoritedStatus)
• void onFollow(User source, User followedUser)
• void onFriendList(long[] friendIds)
• void onUnblock(User source, User unblockedUser)
• void onUnfavorite(User source, User target,
Status unfavoritedStatus)
• void onUserListCreation(User listOwner, UserList list)
• void onUserListDeletion(User listOwner, UserList list)
• void onUserListMemberAddition(User addedMember,
User listOwner, UserList list)
• void onUserListMemberDeletion(User deletedMember,
User listOwner, UserList list)
• void onUserListSubscription(User subscriber,
User listOwner, UserList list)
• void onUserListUnsubscription(User subscriber,
User listOwner, UserList list)
• void onUserListUpdate(User listOwner, UserList list)
• void onUserProfileUpdate(User updatedUser)
35
36. Linked Data & Semantic Web Technology
Example of user Streaming API
1. import twitter4j.*;
2. public class UserStreamListenerImpl implements UserStreamListener {
3. public void onStatus(Status status) {
4. System.out.println("onStatus @" + status.getUser().getScreenName()
5. + " - " + status.getText());
6. }
7. public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
8. System.out.println("Got a status deletion notice id:"
9. + statusDeletionNotice.getStatusId());
10. }
11. public void onDeletionNotice(long directMessageId, long userId) {
12. System.out.println("Got a direct message deletion notice id:"
13. + directMessageId);
14. }
15. public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
16. System.out.println("Got a track limitation notice:"
17. + numberOfLimitedStatuses);
18. }
19. public void onScrubGeo(long userId, long upToStatusId) {
20. System.out.println("Got scrub_geo event userId:" + userId
21. + " upToStatusId:" + upToStatusId);
22. }
23. public void onStallWarning(StallWarning warning) {
24. System.out.println("Got stall warning:" + warning);
25. }
26. public void onFriendList(long[] friendIds) {
27. System.out.print("onFriendList");
28. for (long friendId : friendIds) {
29. System.out.print(" " + friendId);
30. }
31. System.out.println();
32. }
36
39. Linked Data & Semantic Web Technology
Example of user Streaming API
1. import twitter4j.*;
2. public final class PrintUserStream {
3. public static void main(String[] args) throws TwitterException {
4. TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
5. twitterStream.setOAuthConsumer(TwitterAccessToken.consumerKey,
6. TwitterAccessToken.consumerSecret);
7. twitterStream.setOAuthAccessToken(TwitterAccessToken.loadAccessToken());
8. UserStreamListenerImpl listener = new UserStreamListenerImpl();
9. twitterStream.addListener(listener);
10. twitterStream.user();
11. }
12. }
39
40. Linked Data & Semantic Web Technology
Site Streams
• Overview
– services to receive real-time updates for a large number of users
– for any user who has granted OAuth access to your application
– Site Streams is currently in a limited beta. Access is restricted to
whitelisted accounts.
• Endpoint
– GET site
• Types of messages
– the same types of messages streamed to User Stream
• Protected data
– to provide both public and private data in a single stream
– meticulous about what data they display
• Connecting
– to only receive messages for users who have approved OAuth
access to your application
– to consume significant (> 1 Mbit/sec) bandwith
• Limits
– must be respected during the beta period
• limit the follow count to 100 users per stream while connecting
• open no more than 25 new connections per second
40
41. Linked Data & Semantic Web Technology
Site Stream Messages Types
• Envelopes (for_user)
– an additional wrapper to identify the target of each
message except for blank keep-alive lines
• Control messages (control)
– a control message which may be used to modify the
Site Streams connection without reconnecting for new
Site Streams
41
42. Linked Data & Semantic Web Technology
Twitter4J Classes for site Streaming
• TwitterStream Interface
– Methods
• StreamController site(boolean withFollowings, long[] follow)
– Site Streams, a new feature on the Streaming API, is now available for beta testing.
• SiteStreamsListener Interface
– Methods
• void onBlock(long forUser, User source, User blockedUser)
• void onDeletionNotice(long forUser, long directMessageId, long userId)
• void onDeletionNotice(long forUser,
StatusDeletionNotice statusDeletionNotice)
• void onDirectMessage(long forUser, DirectMessage directMessage)
• void onDisconnectionNotice(java.lang.String line)
• void onException(java.lang.Exception ex)
• void onFavorite(long forUser, User source, User target, Status favoritedStatus)
• void onFollow(long forUser, User source, User followedUser)
• void onFriendList(long forUser, long[] friendIds)
• void onStatus(long forUser, Status status)
• void onUnblock(long forUser, User source, User unblockedUser)
• void onUnfavorite(long forUser, User source, User target,
Status unfavoritedStatus)
• void onUnfollow(long forUser, User source, User unfollowedUser)
• void onUserListCreation(long forUser, User listOwner, UserList list)
• void onUserListDeletion(long forUser, User listOwner, UserList list)
• void onUserListMemberAddition(long forUser, User addedMember,
User listOwner, UserList list)
• void onUserListMemberDeletion(long forUser, User deletedMember,
• User listOwner, UserList list)
• void onUserListSubscription(long forUser, User subscriber, User listOwner,
UserList list)
• void onUserListUnsubscription(long forUser, User subscriber, User listOwner,
UserList list)
• void onUserListUpdate(long forUser, User listOwner, UserList list)
• void onUserProfileUpdate(long forUser, User updatedUser)
42
43. Linked Data & Semantic Web Technology
Example of site Streaming API
1. import twitter4j.*;
2. public class SiteStreamsListenerImpl implements SiteStreamsListener {
3. public void onStatus(long forUser, Status status) {
4. System.out.println("onStatus for_user:" + forUser + " @"
5. + status.getUser().getScreenName() + " - " + status.getText());
6. }
7. public void onDeletionNotice(long forUser,
8. StatusDeletionNotice statusDeletionNotice) {
9. System.out.println("Got a status deletion notice for_user:" + forUser
10. + " id:" + statusDeletionNotice.getStatusId());
11. }
12. public void onFriendList(long forUser, long[] friendIds) {
13. System.out.print("onFriendList for_user:" + forUser);
14. for (long friendId : friendIds) {
15. System.out.print(" " + friendId);
16. }
17. System.out.println();
18. }
19. public void onFavorite(long forUser, User source, User target,
20. Status favoritedStatus) {
21. System.out.println("onFavorite for_user:" + forUser + " source:@"
22. + source.getScreenName() + " target:@" + target.getScreenName()
23. + " @" + favoritedStatus.getUser().getScreenName() + " - "
24. + favoritedStatus.getText());
25. }
26. public void onUnfavorite(long forUser, User source, User target,
27. Status unfavoritedStatus) {
28. System.out.println("onUnFavorite for_user:" + forUser + " source:@"
29. + source.getScreenName() + " target:@" + target.getScreenName()
30. + " @" + unfavoritedStatus.getUser().getScreenName() + " - "
31. + unfavoritedStatus.getText());
32. }
33. public void onFollow(long forUser, User source, User followedUser) {
34. System.out.println("onFollow for_user:" + forUser + " source:@"
35. + source.getScreenName() + " target:@"
36. + followedUser.getScreenName());
37. }
43
44. Linked Data & Semantic Web Technology
Example of site Streaming API
38. public void onUnfollow(long forUser, User source, User followedUser) {
39. System.out.println("onUnfollow for_user:" + forUser + " source:@"
40. + source.getScreenName() + " target:@"
41. + followedUser.getScreenName());
42. }
43. public void onDirectMessage(long forUser, DirectMessage directMessage) {
44. System.out.println("onDirectMessage for_user:" + forUser + " text:"
45. + directMessage.getText());
46. }
47. public void onDeletionNotice(long forUser, long directMessageId, long userId) {
48. System.out.println("Got a direct message deletion notice for_user:"
49. + forUser + " id:" + directMessageId);
50. }
51. public void onUserListMemberAddition(long forUser, User addedMember,
52. User listOwner, UserList list) {
53. System.out.println("onUserListMemberAddition for_user:" + forUser
54. + " member:@" + addedMember.getScreenName() + " listOwner:@"
55. + listOwner.getScreenName() + " list:" + list.getName());
56. }
57. public void onUserListMemberDeletion(long forUser, User deletedMember,
58. User listOwner, UserList list) {
59. System.out.println("onUserListMemberDeletion for_user:" + forUser
60. + " member:@" + deletedMember.getScreenName() + " listOwner:@"
61. + listOwner.getScreenName() + " list:" + list.getName());
62. }
63. public void onUserListSubscription(long forUser, User subscriber,
64. User listOwner, UserList list) {
65. System.out.println("onUserListSubscribed for_user:" + forUser
66. + " subscriber:@" + subscriber.getScreenName() + " listOwner:@"
67. + listOwner.getScreenName() + " list:" + list.getName());
68. }
69. public void onUserListUnsubscription(long forUser, User subscriber,
70. User listOwner, UserList list) {
71. System.out.println("onUserListUnsubscribed for_user:" + forUser
72. + " subscriber:@" + subscriber.getScreenName() + " listOwner:@"
73. + listOwner.getScreenName() + " list:" + list.getName());
74. }
44
45. Linked Data & Semantic Web Technology
Example of site Streaming API
75. public void onUserListCreation(long forUser, User listOwner, UserList list) {
76. System.out.println("onUserListCreated for_user:" + forUser
77. + " listOwner:@" + listOwner.getScreenName() + " list:"
78. + list.getName());
79. }
80. public void onUserListUpdate(long forUser, User listOwner, UserList list) {
81. System.out.println("onUserListUpdated for_user:" + forUser
82. + " listOwner:@" + listOwner.getScreenName() + " list:"
83. + list.getName());
84. }
85. public void onUserListDeletion(long forUser, User listOwner, UserList list) {
86. System.out.println("onUserListDestroyed for_user:" + forUser
87. + " listOwner:@" + listOwner.getScreenName() + " list:"
88. + list.getName());
89. }
90. public void onUserProfileUpdate(long forUser, User updatedUser) {
91. System.out.println("onUserProfileUpdated for_user:" + forUser
92. + " user:@" + updatedUser.getScreenName());
93. }
94. public void onBlock(long forUser, User source, User blockedUser) {
95. System.out.println("onBlock for_user:" + forUser + " source:@"
96. + source.getScreenName() + " target:@"
97. + blockedUser.getScreenName());
98. }
99. public void onUnblock(long forUser, User source, User unblockedUser) {
100. System.out.println("onUnblock for_user:" + forUser + " source:@"
101. + source.getScreenName() + " target:@"
102. + unblockedUser.getScreenName());
103. }
104. public void onDisconnectionNotice(String line) {
105. System.out.println("onDisconnectionNotice:" + line);
106. }
107. public void onException(Exception ex) {
108. ex.printStackTrace();
109. System.out.println("onException:" + ex.getMessage());
110. }
111. }
45
46. Linked Data & Semantic Web Technology
Example of site Streaming API
1. import twitter4j.*;
2. public class PrintSiteStream {
3. public static void main(String[] args) throws TwitterException {
4. TwitterStream twitterStream = new TwitterStreamFactory().getInstance();
5. twitterStream.setOAuthConsumer(TwitterAccessToken.consumerKey,
6. TwitterAccessToken.consumerSecret);
7. twitterStream.setOAuthAccessToken(TwitterAccessToken.loadAccessToken());
8. SiteStreamsListenerImpl listener = new SiteStreamsListenerImpl();
9. twitterStream.addListener(listener);
10. long[] followArray = {52401019, 1476571, 822659, 20606823, 18304822};
11. twitterStream.site(true, followArray);
12. }
13. }
46