Apache Kafka can be used to create an interoperability layer for electronic health systems in Zambia. As a distributed streaming platform, Kafka allows different health information systems to communicate asynchronously using topics that correspond to HL7 message templates. This provides a standardized, decoupled, and scalable approach to sharing data between systems in a plug-and-play manner. By storing all messages on topics in Kafka, the system also enables a single source of truth and audit trail for all health data transactions in the country.

1. Seamless, decoupled, multi-directional and
standards-based interoperability in the E-Health
Space
Ifunga Ndana
MIKA Hotel Kabulonga
11th July 2019
E-Health Interoperability
Layer through Kafka

2. Interoperability Layer
“An interoperability layer is a system that
enables simpler interoperability between
disparate information systems. In the context of
Electronic Health, these are Health Information
Systems (HISs) such as a Client Registry, Lab
Information System, Facility Registry and a
shared Health Record.” – https://ohie.org
2

3. Quick Outline
•We will begin by discussing the Interoperability
Layer based on the OpenHIE architecture as a
reference point.
•Following this we will touch on how a technology
called Apache Kafka can be applied to achieve
seamless, decoupled, multi-directional and
standards based interoperability in an effective
manner.
3

4. Architecture & Principles
“The fundamental basis of interoperability is in harmonizing the context with
which information is collected in so that it’s broadly reusable to a much
larger set of stakeholders.”
“The OpenHIE architecture supports interoperability by creating a framework
that maximally leverages health information standards, enables flexible
implementation by country partners, and supports interchangeability of
individual components.”
https://ohie.org/architecture/
4

5. An Interoperability Layer must:
•Harmonize the context with which information is collected;
•Utilize Health information standards (HL7, LOINC, ICD10);
•Be broadly reusable (plug & play);
•Cater to a large set of stakeholders;
•Allow for Flexible/independent implementation within each
system;
•Allow for interchangeability of individual components; and
•Allow for identification (Authorization/Authentication);
5

6. What is Apache Kafka®?
•Apache Kafka® is a distributed data streaming platform.
•It was originally developed by LinkedIn before being
donated to the Apache Foundation as an open-source
project.
•The project aims to provide a unified, high-throughput,
low-latency platform for handling real-time data feeds
•Kafka has several features which make it well suited for
use as an interoperability layer.
6

7. How does Apache Kafka® work?
Kafka leverages two system design principles for its core implementation: Message Queuing
and The Publisher/Subscriber model
“Message queues allow different parts of an ecosystem (e.g. Microservices / disparate
systems) to communicate and process operations asynchronously. A message queue
provides a buffer which stores messages for a defined period, and endpoints (connections)
that allow software components to communicate with the queue in order to send and receive
messages.” – Amazon
“The Publisher-Subscriber pattern enables an application to automatically announce events to
multiple interested consumers asynchronously, without coupling the senders to the
receivers.” - Microsoft
7

8. Apache Kafka® - Topics
• In Kafka, one or more Message Queues can be defined. These Message Queues are called
Topics. All messages must be written (Published) to and also read (Subscribed) from
specific Topics
• One or multiple systems can Subscribe to one or multiple Topics. One or multiple systems
can Publish to one or multiple Topics. In short communication can be multi-directional.
• Any systems in the space will only need to interface with Kafka (the Interoperability Layer) and
not each other directly. These systems will be completely decoupled.
• This way the development, upgrade and roll-out of each system is completely independent
of each other.
• Furthermore, this allows the Interoperability Layer to be plug and play as any number of
systems can be added to the ecosystem as long as they adhere to the Topic Message
Template.
8

9. HL7 2X Messaging Standards
• Kafka’s use of Topics is well suited to the HL7 2X Messaging Standard developed by an
organization called Health Level Seven International
• HL7 has standard Message Templates, each covering specific domains within the E-Health
space. For example, the message template for a Patient Registration is ADT_A04 while the
message template for a Lab Order is OML_O21.
• In the Interoperability Layer, the HL7 Message Templates would be used to define Topics
• One or more systems would Subscribe to relevant Topics to automatically read all
incoming messages on the queue.
• Similarly one or more systems would Publish messages to specific Topics as well, these
messages would then be automatically broadcast to ALL subscribing systems without
need for manual polling or system notifications
• With this in place, and using reliable Patient Identification, a system such as DISA could post
a Lab Result to the relevant Topic (ORL_O22). Following this, SmartCare (and indeed any
other subscribed system) would automatically be made aware of a new message on the
ORL_O22 Queue and process this it, updating the client record.
9

10. HL7 2X Messaging Standards
• HL7 Messaging Templates would allow each system in the space to have
unique/independent underlying implementations. The only requirement is that
outgoing messages be encoded in standard HL7 Message Templates.
• With this systems such as SmartCare or ELMIS would not have to modify
their underlying data structures, rather, only need to expose their information
in a standard HL7 Message Template that any other system can translate.
• Similarly, systems such as SmartCare or ELMIS could have their underlying
data structures or indeed technology completely rewritten from the ground up
without compromising the ecosystem granted that the correct HL7 Message
Template is adhered to.
• In short, adopting HL7 Messaging (as well as other standards such as LOINC,
ICPC2 & ICD10) adds both consistency and flexibility to the entire E-Health
ecosystem
10

11. HL7 2X Messaging Standards
• HL7 Message Templates are self-contained. Meaning each message has enough meta-data to satisfy
a request without need for follow-up calls.
• Specifically each HL7 Message Template has a MSH (Header) section
• The Header Includes:
• The sending application e.g. SmartCare
• The sending application version e.g. August 2019 Release
• The sending facility e.g. Chilenje 1st Level Hospital (System level ID for consistency)
• The sending timestamp
• Furthermore, most messages include a PV1 (Patient Visit), a PID (Patient Identification) and a PID1
(Patient Demographics) section
• In short, if a system like DISA got an HL7 Lab Order Message from SmartCare it would not need to
call SmartCare for additional information such as the gender or age of a client as all of this would be
provided upfront.
• In short HL7 is very efficient for use in Interoperability
• HL7 2X is a separate topic which I encourage further reading on – Carepoint Health
11

12. HL7 Messages - Sample
MSH|^~&|OF|Chemistry|AM|Automation|200309060825||OML^O21^OML_O21|msgOF101|T|2.5.1|123||||USA||EN
PID|1||12345^5^M10^Memphis_Hosp^PI||EVERYMAN^ADAM^^JR^^^L||19800101|M
PV1|1|O|Ward|||||||||||||||12345
ORC|NW|||666^gastric|||||200309060824|222221^NURSE^NANCY||||||||||| Entero-gastric^^^^^^FI^^^EG02
TQ1|||||||||A
OBR||555_1^chemistry||GLUC^GLUCOSE^L||||||1234^BLEEDER|S|||||222222^PHYSICIAN^^^^DR |821
SPM|1|123456781^gastric ||SER|||||||P||||||200309060735|200309060821||||||||1
ORC|NW|||666^gastric|||||200309060710|222221^NURSE^NANCY||||||||||| Entero-gastric^^^^^^FI^^^EG02
TQ1|||||||||A
OBR||555_2^chemistry||GLUC^GLUCOSE^L||||||1234^BLEEDER|S||||| 222222^PHYSICIAN^^^^DR|821
SPM|1|123456782^gastric||SER|||||||P||||||200309060755|200309060821||||||||1
12

13. Use Case – Lab Orders, Results and Pharmacy Prescriptions
• We have SmartCare which is the National EHR sitting in multiple facilities
• During the provision of care Lab Orders would be created within SmartCare
• Following this, we would expect these Lab Orders (with all relevant
information) to be shared with a dedicated Lab Information System, in this
case DISA
• Lab Technicians should be able to see all incoming Lab Orders in DISA, and
based on the availability of samples, run various tests
• These Lab Orders should be entered in DISA and shared back to SmartCare
• It would be beneficial to monitor the flow of all lab orders and lab results at
higher aggregation levels
• For this, a dedicated Statistics Aggregation Application called Lab Trans
would be expected to be aware of all these transactions
13

14. Use Case – Lab Orders, Results and Pharmacy Prescriptions
14

15. Benefits
• Multiple systems can co-exist in the same space.
• These systems all speak the same "language" due to the use of standardized
HL7 2X messages
• New systems can be added to the space without major disruption, this is
because the systems are decoupled
• Full transparency at national aggregation level (MOH has visibility on all
shared messages)
• If any system is down, the other systems continue to function regardless
• Backlogs (for both the sending & receiving of messages) are resolved
automatically when the system comes back online
• Every system can be extended, upgraded, rewritten and deployed
independently of each other.
15

16. Benefits - Single source of truth and audit-trail
• All messages coming to all topics defined in Kafka can have a set storage
period applied. While the default is 7 days this can be set to several years or
even infinity. Alternatively this data can be configured to be written to a
conventional database such as MySQL or PostgreSQL for backup through
“Kafka Connectors” (a means of directly outputting Kafka messages to
Database Tables)
• This means that through Kafka the Ministry of Health and relevant
stakeholders could have access to every message sent by every system in
one location.
• Furthermore this allows newly introduced or existing systems to access all
historic data for backlog or ETL/re-importing purposes.
16

17. Benefits - A mature technology with robust support and
documentation
• Kafka is used in production by many of the worlds largest organisations
(LinkedIn, Uber, Netflix, Twitter etc)
• Kafka can be easily integrated into existing systems through libraries
developed by Confluence (official Kafka maintainer) and organisations like
Microsoft and Oracle.
• Once integrated into systems developer effort would be dedicated to the
creation and parsing of HL7 Message Templates only. Polling, Reading,
Writing, Synchronization and Offsets are all handled by Kafka leaving
implementors to focus more on the messages rather than on low level
technology.
• Kafka is free to use and open-source with no need for licensing fees
• Kafka can be hosted on-premise, within a Data-Centre (ZNDC) or on the
cloud.
17

18. Code Sample – Basic Producer in C#
18https://github.com/confluentinc/confluent-kafka-dotnet

19. Code Sample – Basic Consumer in C#
19
https://github.com/confluentinc/confluent-kafka-dotnet

20. Code Samples – Other Languages
20
•Code samples for more languages Java, Kotlin & C# can
be found here: https://github.com/confluentinc/examples

21. Thank you
•Questions ??
21