1. IntroductionEvent processing is getting more popular concept in several sectors. It can be thekey concept that should be focused in the near feature technology trends. So weprepared this booklet for to give an idea about event processing, usages and sampleimplementations with our Complex Event Processing product EVAM (Event & ActionManager).Recently we were doing most of analysis on data that remains in the databases, butnow it is changing; it is time to process steaming data, before it goes to storage. Wecall “event” as hot data or we can say that event is moving data, so if we want tocreate real time or near real time architectures we should talk about event.We are facing with big data concepts as well. Currently event producer systems(hardware, &software) produces huge amount of data and cycle of data is becomingas follows;We were waiting to process data after it comes to the storage areas such asdatabases, but now we have capabilities to process it as it is produced. This is newera of processing the events that will be data after some time.
2. What is Event? An event is any detectable occurrence in a particular system or domain indicating a change in the state of the business, something that happens. Countless events going on constantly everywhere. We react to events every day. An event (business or system) may signify a problem or impending problem, an opportunity, a threshold, or a deviation. These events may be happeningacross the various layers of an organization as sales leads, orders or customerservice calls. Or, they may be news items, text messages, social media posts, stockmarket feeds, traffic reports, weather reports, or other kinds of data. When youtweet something that is actually an event. Or when you share a photo on Facebook.Or when you use your credit card to buy something. Or when you call someone, sendtext messages. The list is countless.There are different types of events that we can group with a property suchfrequency. Some of them are repetitive. Happens with a certain interval. Lots ofthem happen randomly. Or we can group events by domain. Some examples for somespecific domains are as follows:Bank Events:Credit card usage, bankcard usage, shopping cancellation, money order, electronicfunds transfer, open/close account, cash sale, installment sale and much more.E-Business Events:New Registering customer (register events), login, purchase, shopping basketupdates, shopping card updates, viewing different products of same category,customer logins from different locations, fraud detection.Telecommunication Events:Postpaid subscribers payments from a bank payment method, tariff changes,account activation, voice calls by prepaid subscribers, voice calls by postpaidsubscribers, voice package usages of subscribers, voice pack, sms or data usages ofsubscribers, voice calls or receiving sms from rival Telco firmOnline Gaming Events: Player’s login, player’s movements, player’s strategies.
3. Where are they come from? An event can originate from any suitable platform. Today, huge numbers of events come from databases, software applications, devices and any other platforms. Directly humans and systems or devices themselves create events.Hardware event producersHardware producers are used extensively in a number of application areas:- Medical equipment and personal body sensors (for example, heart-rate monitors)- Device management (computer systems or industrial equipment)- Defense and military applications- Security applications- Traffic management systemsThe archetypal hardware event producer is a sensor that generates events thatreport on one or more aspects of the physical environment in which it is situated,for example, a smoke detector. A sensor can be packaged as a discrete piece ofhardware, such as the smoke detector example; or it can be embedded into anotherpiece of equipment, for example, a sensor that detects the fan speed on a computermotherboard. The simplest kind of sensor reports on just one aspect of itsenvironment, for example:- Motion of the sensor itself including vibration- Tilt or angle of orientation of the sensor- Rotation of a rod attached to the sensor- Temperature- Humidity- Light (intensity or color)- Infrared or radio waves- Sound (intensity of frequency)- Air (or other gas) pressure- Physical pressure applied to the sensor itself.More sophisticated detectors use one or more of the physical detection mechanismsto make specific observations or look for particular kinds of occurrence.Here is a small list of examples:
- A sensor that detects motion external to itself, such as a passive infrared (PIR)sensor. This can be used to detect the presence of people in the vicinity of thesensor, for example, when reporting on room occupancy or when looking forIntruders.- A sensor that detects whether the door on the casing of equipment is open orclosed.- An RFID reader used to detect the presence of an RFID tag. This can be used insupply chain or many other application areas.- A seismometer used to detect and report on earthquakes or nuclear tests.- A traffic speed detector. Apart from their obvious use in penalizing speedingdrivers, speed detectors can also be used in intelligent traffic management systems.- GPS location devices. These are used in a wide variety of tracking and locationaware services.Cameras (both still and video), microphones, telephones, and radio receivers can alsobe viewed as event producers because the data that they produce can be processedby event processing applications. For example, you could have a security applicationthat processes frames coming from a video camera looking for the presence ofunauthorized personnel in a secure area.Software event producersAlthough software is often associated with a hardware producer, some eventproducers are made up of software only.The first category consists of simulated sensors. Simulated sensors are used whenthe entire external system is itself a simulation, for example, a flight trainingsimulator or virtual reality game, and they can also be used to stand in for a realpiece of hardware when testing an event processing application.An event producer could be a first-class part of a software application. By this wemean that it is a piece of application logic that explicitly generates an event objectand submits it to the event processing network. This may happen as a result of ahuman interaction but sometimes events can be generated less directly. Forexample, a financial trading system might include a settlement application thatautomatically generates payment events. This application uses a programminginterface to submit the event.Events can be produced indirectly by a technique known as instrumentation. Herethe events are not generated by application code itself, but instead are produced bysoftware that is monitoring the application, looking for noteworthy activity. Thiskind of producer is sometimes called a monitor or probe. A wide range of eventscould be viewed as noteworthy, from tracing program start-up and shutdown ortracing function calls within the application, through detecting updates that theapplication makes to its data, reporting computer performance statistics, orspotting and reporting on hardware or software errors if they occur.
Instrumentation can be provided by the operating system or the container that runsthe application, or by database or messaging middleware that the application uses.Examples include workflow engines which can generate events when a particularworkflow goes through a state transition, security subsystems which can generatealerting events when they detect attempted security violations, and messagequeuing systems which can generate events when the number of messages waiting ina queue exceeds a certain threshold.More recently web-based feeds that use Atom Syndication Format or Really SimpleSyndication (RSS) have emerged. Literally tens of thousands of them are availableon the Internet.Human interactionSome events are generated directly by human interaction, albeit with a bit ofsoftware and hardware assistance. Human interaction can be facilitated by anapplication program with a user interface that in effect allows the user to enter theevent. In our example we could imagine that each store has a web application, andstore personnel use a form-style interface to place a delivery order request.Events can also be generated using a verification or payment device, for example,the delivery confirmation produced by the driver’s handheld device in ourapplication, or a purchase event being generated by a till in a retail store.Some producers detect our presence, for example, the swiping of an identity card(or having a personal RFID tag scanned) when entering a secure area, the use of anNFC (near field communications) tag to go through a ticket barrier on publictransport or pass through immigration control. Instant messaging applications (andincreasingly telephony applications) can produce presence events that indicate whena user has turned a computer or handheld device on or off, and these presenceevents sometimes include information about the user’s location or actions.This brings us to our next area, social communications. As well as providing itsregular web browser interface, the popular Twitter Internet service offers RSSfeeds that can be used to communicate presence (and other) events. Events can alsobe produced from other social networking applications.
4. Why Event Processing? If event processing is such a great idea, why hasn’t everyone been doing it all along? Event processing is underutilized partly because relatively little data on current business events has been available in digital form until recently. In the past, many events either were undetected or were detected but not reporter in digital form that could be sent over anetwork or manipulated by computer. Now, more events are detected andrepresented electronically, although, unfortunately, many are still not readilyaccessible to people, devices, or IT systems that could benefit from them.The amount of available event data is rapidly expanding because of the decreasingcost and increasing speed of computers and networks, and the unifying power ofWorld Wide Web and its communication standards. We are blessed with anexplosion of event “streams” flowing over corporate networks – data from websites,enterprise application systems, e-mail systems, cell phones, RFID readers, GPSsystems, and a variety of other sensors and devices. This Wealth of event data willgrow as the cost of the relevant technologies continues to drop and companiescreate new sources of event data in their operations and the outside environment.Our challenge is to make better use of this data.The other reason that formal event processing has not been widely used in the pastis that competition and customer demands were less urgent. Companies had moretime to respond to events than they have today. A person driving 30 km per hourdoesn’t need as mush advance warning of upcoming curves or obstructions on theroad as a person driving 60 km per hour. Companies today are operating at a fasterpace, so early notification of emerging business threats and opportunities areevermore important. Companies that know how to leverage event processing have anadvantage over those that don’t.
5. How Does Event Processing Work?Event processing is a method of tracking and analyzing (processing) streams ofinformation (data) about things that happen (events), and deriving a conclusion fromthem.Event processing starts with capturing events from source systems. Once you havebuilt the right architecture any system can be used as an event source. Web sites,credit cards systems, GSM networks and Call Centers are just a few examples forevent sources in real world. EVAM has both hardware and software components thatlisten live systems to capture events.The next step, Parsing&Filtering is responsible for generating business events fromraw events. In this step filtering, parsing and formatting rules are applied to rawevents before sending them to processing engine.An Event carries vital information but we have critical information resides in othersystems and is not carried with event itself. In Enrichment phase, business eventsare enriched with historical data taken from external sources such as relationaldatabases, NoSql systems, web services etc…Event processing step is the heart of the system. It processes streaming eventsand decides to take action exactly when appropriate conditions are met. In EVAM
world, event processing engine acts according to defined scenarios. Scenarios aredrawn via drag&drop designer reflecting business needs. Scenarios can be seen asrulesets but they have capability act according to event history and customercontext. All event processing is done in-memory meaning that you can processevents in real-time.You have captured, parsed, enriched and processed events.. All these steps are donefor taking actions. Sending sms/mail, calling a web service, mentioning someone in atweet, liking some page in Facebook are all examples for actions. Actions are touchpoints of event processing with other systems. You can either touch your customerby sending sms or generate an alarm in a monitoring system via an interface.All these critical steps should be monitored from an centralized system both foroperational and reporting purposes.6. Data – Database / Event - Event ProcessingA recent question came up regarding the difference between “events” and “data”.Data is something that is your primitive entity that holds business information in atightly coupled SOA or database centric architecture. Events are something carriesaround your business data in an EDA. Data is derived from events. Customer data?Derived from customer registrations and associated checks – all events. Amountowed? Derived from purchase events. And so forth.In Complex Event Processing, we are processing this “event data” as it occurs(within the limitations of our system latency, of course!). Typically this event data iscorrelated with previous events, which can be either “relatively recent” (andconsidered as stored events or event objects) or “historic” (and considered astraditional data), although these are necessarily imprecise concepts. Access tohistoric data is typically carried out in CEP tools through good-old-database-services: calling the operational data store associated with legacy systems, forexample.Event is real-time. So the information carried with them is hot data.
7. Event Processing In Business (Real Time Marketing, Customer Experience Management, Fraud Detection, Sensor Alarms)Business world is a fast-paced environment and becomes harder day-by-day. So,organizations must adapt this environment faster than rivals. Organizations need touse more specialized software applications depending on some selection criteriasuch as time, speed, budget etc.Event processing makes dramatic improvements in business processes and ITsystems possible. It has a direct, tangible impact on the lives of businesspeople.Event processing changes the way they do their jobs by giving them better visibilityinto what is happening in their company and its external environment. It alsoimproves a companys reaction time to unforeseeable situations, reduces the end-to-end elapsed time of business processes, and improves the quality and availabilityof information.From this point; EDM, which is the new trend, has some advantages over traditionaldirect marketing campaigns:Higher response rateEDM typically has higher response rates than traditional DM campaigns. Not only isthe client more perspective for your offer because you target him at theappropriate time and through the channel of choice of client, you also gain bysending out less communication.Higher impactNot only do you leverage the response rate, you also have more impact on the clientresulting in higher profit.Possibility of testingIn contrast to traditional campaigns where testing is either very expensive ornearly close to real campaign processes, EDM lets you test new ideas for scenariosrather quickly because once the scenario is ready you can select a limited numbersof clients daily for testing, or test manually with hypothetical.
Ongoing adjustmentBecause you select a target group and send out messages every day, you can monitorand adjust both selection criteria, scenario and message content daily. Compare thisto a traditional campaign where you can adjust these variables let’s say once a year.Homogenous loadWith EDM, you level out the work load (both financial as in terms of humanresources). You divert the peaks of load in the startup-phase of a new traditionalcampaign to daily operations.8. Future Of Event ProcessingSome people say that event processing will be the next big development incomputing; others say that event processing is old hat with nothing new.CEP starts becoming an enabler for real time business intelligence in the last fewyears, the technologies for implementing real time BI solutions have evolveddrastically making it a viable alternative in the enterprise. A clear example is theupcoming Microsofts PowerPivot stack that is able to process millions of records ina highly optimized multidimensional store that can be accessed from both Excel andSharePoint environments.The missing component of real time BI solutions is the mechanisms for collectingand processing the data in highly efficient ways. This is where CEP could shine. MostCEP engines are optimized for processing a large number of events using continuousquerying mechanisms. In that sense, CEP technologies should become an essentialcomponent of real time solutions in order to make these type of solutions a viablealternative in theBig enterprise.CEP engines go mobileTraditionally, CEP applications are based on a server-centric model that isresponsible for hosting, managing and scaling the core components of the CEPengines. Even though a large number of CEP scenarios can be implemented using thismodel, there are native capabilities of CEP engines such as the continuous queryengines or adapters that can be very effective on mobile applications. For instance,consider a traditional RIFD scenario on which a mobile reader is capturingthousands of events that need to be filtered based on different patterns. On this
scenario, w e could read the events using a CEP adapter for the mobile RFID readerand filter them using a CEP continuous query engine. This post does not intend to beformalized list predictions about the CEP market. Most of the thoughts listed thereare based on tendencies that I have seen in different CEP projects throughoutrecent years.9. Complex Event Processing with EVAM (Event & Action Manager) EVAM® is a r e a l time event-processing solution capable of s e n s e , f i l t e r , e n r i c h a n d response to events as it is produced. It can process millions of events in seconds and can create actions that can be customized depending on the requirements. EVAM is a highly elastic and easy to use event processing solution empowered with the ability to react to potentialopportunities or risks in real-time and at an unprecedented scale.Event loops became a central design principle in programming following theintroduction of graphical user interfaces. Programs listening to various events wereprincipally designed to accommodate unpredictable users who might do anything infront of their computers at any given time.The framework of our solution consists of three main components: Scenario Studio,Event-listeners, and Processing Engine.Recognition of patterns in event-streams, enrichment of data by connecting to otherdata sources, decision trees and all the governing logic and rules are designed usingsimple drag&drop techniques at our Scenario Studio.Event-listeners are practical low impact components that are installed at variousevent sources to capture and send relevant data to the Processing Engine(s).The scenarios are compiled by the Scenario Studio and loaded up to a cluster ofengines for high velocity processing. The Event-listeners stream data to theProcessing Engine cluster and our innovative architecture ensures persistence of highvolumes and both vertical and horizontal elasticity from a few hundred to billions ofevents per day.
Event & Action Manager (EVAM) allows you to abstract and correlate meaningfulbusiness information from the events and data flowing through your informationsystems, and take appropriate actions using scenarios. By detecting patterns withinthe real-time flow of events, EVAM can help you to detect and understand unusualactivities as well as recognize trends, problems, and opportunities EVAM publishesthis business-critical information in real time to your critical enterprise systems,dashboards or create response actions. With EVAM you can predict the needs ofyour customers, detect frauds, make faster decisions, and take faster action.There are many application areas for EVAM such as real time offer management,fraud detection, real time alert generations etc.Next Generation Marketing Solution concept that aims to take necessary action bycatching the demand when it emerges is with you thanks to Event&ActionManager(EVAM). Intellica, known by the power of its intellectual capital and itsmastery in high technology in Business Intelligence and Customer RelationsManagement presents the Next Generation Event Processing solution toorganizations.EVAM provides realization of sale and event-driven real-time marketing activities,especially for sectors of finance, retailing and telecommunication. EVAM broadensmarketing departments’ horizons by integrating into data warehouses and customerrelations management systems of organizations easily.Product technically capable of processing billion of events from many event sourcesystems and expand horizontally with new clusters. There are seamless integrationcapabilities with event sources in both pull and push methods.EVAM is, it is business centric tool; with drag and drop type of user interface, atypical power business user is able to implement new event driven scenarios in EVAMwith minimum IT support. This give opportunity to organizations decrease time tomarket dramatically for new marketing ideas.There is a paradigm shift happening from business intelligence to operationalintelligence. EVAM is one of the very few proven and pioneering products in thismarket.
EVAM is one of the newest products in the CEP market with its unique approach to eventprocessing. It is possible to develop deterministic and nondeterministic scenariostogether.10 Reason to Use EVAM1. Your application might need to identify and react to certain situations (either good orbad) as they occur. An event-driven approach, where changes in state are monitored asthey happen, lets an application respond in a much more timely fashion than a batchapproach where the detection process runs only intermittently.2. Event processing can give you a way of extending an existing application in a flexible,non-invasive manner. Rather than changing the original application to add the extrafunction, it’s sometimes possible to instrument the original application by adding eventproducers to it (for example, by processing the log files that it produces). The additionalfunctionality can then be implemented by processing the events generated by these eventproducers.3. Intermediary event processing logic can be separated out from the rest of theapplication. This can allow the application to be adapted quickly to meet new businessrequirements, sometimes by the application business users themselves.4. The application might involve analysis of a large amount of data in order to provide anoutput to be delivered to a human user or another application. This data can be organizedinto streams of events which are then distributed to multiple computing nodes allowingseparate parts of the analysis to be performed in parallel.5. There are potential scalability and fault tolerance benefits to be gained by using anevent-driven approach. An event-driven approach allows processing to be performedasynchronously, and so is well suited to applications where events happen in an irregularmanner. If event activity suddenly spikes, it may be possible to defer some processing to asubsequent, quieter time.6. Create deterministic or nondeterministic scenarios7. Create more business involvement8. Integrate Decision Engine with Event Processing9. Your application might be naturally centered on events.10. Get high volume event processing capability
Please visit www.evam.com.tr for more information about EVAM or firstname.lastname@example.org to test the product on your site.