![Translating Query Parameters : Select
Contains a comma separated list of fields to return.
A * in the select parameter indicates that all fields should be returned.
For our app to work we turn the * into a list of specific fields so we call the service and use this dataweave to
set a variable called select.
%dw 2.0
output text/plain
import * from dw::core::Objects
var data = if (payload != null) payload[0] else []
var names = if (typeOf(data) as String == “Object” nameSet(data) else []
var select : String = vars.select match {
case select if (select is String and select != ”” and select !=”*”) -> select as String
case “*” -> joinBy(names, ”, “) as String
else -> joinBy(names, “, ”) as String
} default joinBy(names, ”, “) as String
---
select](https://image.slidesharecdn.com/mulesoftlondoncop-febfinal-200303122126/75/MuleSoft-London-Community-February-2020-MuleSoft-and-OData-28-2048.jpg)
![Translating Query Parameters : Filtering
Salesforce has a few curious ways of implementing these filters.
For example instead of contains it uses indexof to find a given string in a field
We translate the filter operators from words into operator characters, i.e. eq -> =
We translate datetimeoffset to xs:date format
var odataFilterToXQFilter = (odatafilter) -> (( odatafilter replace ”eq null” with “= ‘’”
replace /(?i)indexof((’*?|”*?)(w*?)(’*?|”*?),((’*?|”*?)(w*?)(’*?|”*?)))((s*?)ne(/s*?)-1)/
with (“fn:contains(“ ++ $[2] ++ “,” ++ $[4] ++ “)”)
replace “ne null” with “!= ‘’”
replace “ eq “ with “ = “
replace “ ne “ with “ != “
//Further replacements of operators omitted
replace /(?i)(datetimeoffset’)(d{4,4}-d{1,2}-d{1,2})Td{1,2}:d{1,2}:d{1,2}(Z)(‘)/ with |( “xs:date(‘” ++
$[2] ++ $[3] ++ ”’)”)
) splitBy ” “
map (item, idx) -> (
if(names contains item ) “$tran/” ++ item else item
)
) joinBy “ “](https://image.slidesharecdn.com/mulesoftlondoncop-febfinal-200303122126/75/MuleSoft-London-Community-February-2020-MuleSoft-and-OData-29-2048.jpg)
![Translating Query Parameters : Selecting
and Paging
The previous steps have given us a payload that contains the correct full set of records for the given query
parameters.
The final step is to apply the paging parameters to return the correct subset of records and fields to the client.
For this we use the following dataweave on our payload.
%dw 2.0
output application/json
var transactionCount : Number = sizeOf(payload default [])
var rangeStart : Number = if(vars.skip > transactionCount) transactionCount else vars.skip
var lastIndex : Number = vars.skip + (vars.top -1)
var rangeEnd : Number = if(lastIndex >= transactionCount) transactionCount -1 else lastIndex
---
{
“entries” : if(not isEmpty(payload)) ( payload[rangeStart to rangeEnd] map ( //This filters the record set
$ filterObject (( value, key) -> vars.select contains(key))) // This selects the fields to be returned
) else []
}](https://image.slidesharecdn.com/mulesoftlondoncop-febfinal-200303122126/75/MuleSoft-London-Community-February-2020-MuleSoft-and-OData-33-2048.jpg)
Uploaded byPace Integration
MuleSoft London Community February 2020 - MuleSoft and OData
The document details a Mulesoft London Meetup held in February 2020, covering various integration topics such as Async API, API security, and Salesforce integration using OData. It features expert speakers and discusses a solution design for viewing bank account transactions directly in Salesforce using the Mulesoft OData API Kit. Key resources and references for further learning on these integration techniques are provided at the end.
More Related Content
Similar to MuleSoft London Community February 2020 - MuleSoft and OData
![[BDD 2025 - Full-Stack Development] The Modern Stack: Building Web & AI Appli...](https://cdn.slidesharecdn.com/ss_thumbnails/fs-themodernstackbuildingwebaiapplicationswithserverless-251124030844-388cf04f-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Artificial Intelligence] AI for the Underdogs: Innovation for Sma...](https://cdn.slidesharecdn.com/ss_thumbnails/ai-aifortheunderdogsinnovationforsmallbusinesses-251124030839-72a599a4-thumbnail.jpg?width=640&height=640&fit=bounds)
MuleSoft London Community February 2020 - MuleSoft and OData
- 1. MULESOFT LONDON MEETUP. FEBRUARY2020. https://www.paceintegration.com/ connect@paceintegration.com Contact Pace Integration:
- 2. THE COMMUNITY. Signup at https://meetups.mulesoft.com/london/ Opento everyone Learn and share Interactive Build your network
- 3. SLIDESHARE + LINKEDIN Materialavailable afterwards on SlideShare. ENGAGE ON SOCIAL MEDIA J #MULESOFTMEETUP @PACEINTEGRATION
- 4. ASYNC API Hybrid Integration Patterns Centre for Enablement B2B Integration API Enablement –start small, plan big GREAT TOPICS Anypoint Platform DevOps Tooling with Anypoint RAML 1.0 API Led Connectivity Anypoint Platform Crowd Release SaaS Integration Single view of Customer API Security The Connected Enterprise
- 5. WHO WANTS TOBE A…
- 6.
- 7. ADAM MAGUIRE WILSON FAMOUS FACES. SEAN YOUNGMICHAEL JAKEMAN PIYUSH PATEL KA MAN ALEX THEEDOM Miguel Ángel Chuecos
- 8. VIKAS SINGH Integration ProgramManager Tata Consultancy Services A Seasoned Consultant with an interest in Business Agility and Enterprise Architect to bridges the gap between technology and business through automation and integration that unlocks true business value.
- 9. PAWAN GUPTA Integration Architect TataConsultancy Services An Integration Architect involved in the Solution design, architecture, development and implementation. Always keen to explore and learn new technology to give advantage to business.
- 10. 10 Oh Data! MartinGardner Solution Principal at Slalom How I learned to stop worrying and love integrating Salesforce with the Mulesoft Odata API Kit
- 11. ABOUT ME • 16years in CRM • 8 years in Salesforce • Data Integration and migration has been a theme throughout my career • I love data • 6 x Salesforce Certified • Mulesoft Certified Developer Level 1 Mule 4 • https://www.linkedin.com/in/martingardner/ • martin.gardner@slalom.com
- 12. Agenda • ProblemStatement & Solution Design • A very short introduction to OData • Salesforce Connect • Mulesoft OData API Kit • Translating Salesforce OData Query Parameters • Putting it all together • Summary
- 13. Problem Statement As a Salesforceuser I want to view the bank account transactions for the accounts I own from within the Salesforce user interface so that I don’t have to login to the back-office banking system. As a System owner I want to store all accounts’ transactions outside Salesforce so that I avoid exceeding my Salesforce storage limits.
- 14.
- 15. Salesforce External Object Mulesoft Banking Platform HTTPS OData Request HTTPS OData Response SOAPRequest SOAP Response Transaction List User
- 16.
- 17.
- 18. OData (Open DataProtocol) • Defines best practices for building and consuming RESTful APIs • Response Headers • Status Codes • HTTP methods • URL conventions • Media types • Payload formats • Query options • Tracking changes • Defining functions/actions for reusable procedures, and sending asynchronous/batch requests • https://odata.org • Usually used to expose databases as RESTful services.
- 19. OData key features •Feeds : Collections of typed Entries • Entry: Represents a structured record with a key that has a list of Properties • Properties: Primitive or complex types • Links: Connect entries into a hierarchy of related entries and feeds. • Service Operations: Simple service-specific functions • Service Metadata Document • Describes the data source metadata • Allows clients to discover the top-level feeds • Available at the service root URI as Atom or JSON
- 20. OData Operations • Retrievingthe Service Document • GET https://services.odata.org/OData/OData.svc/$metadata • Retrieving feeds • Request • GET /OData/OData.svc/Products HTTP/1.1 Host: services.odata.org accept: application/atom+xml,application/xml DataServiceVersion: 1.0 MaxDataServiceVersion: 2.0 • Response • HTTP/1.1 200 OK Content-Length: 5685 Date: Sat, 27 Feb 2010 20:03:28 GMT Content-Type: application/atom+xml;charset=utf-8 DataServiceVersion: 1.0; <?xml version="1.0" encoding="utf-8" standalone="yes"?> <feed xml:base="https://services.odata.org/OData/OData.svc xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"> <title type="text">Products</title> <id>https://services.odata.org/OData/OData.svc/Products</id> <updated>2010-02- 27T20:03:28Z</updated> <Link rel="self" title="Products" href="Products" /> <Entry> ... </Entry> <Entry> ... </Entry> <Entry> ... </Entry> </feed>
- 21.
- 22. Salesforce Lightning Connect •A framework for viewing, searching and modifying data that is stored outside a Salesforce org. • No need for ETL • Data does not sit in Salesforce and consume data storage on platform • Use when • You have large volumes of data that you don’t want to copy to Salesforce • You need small amounts of data at any one time • You need real-time access to the latest data • You want to display the data in your Salesforce org. • Salesforce Lightning Connect has connectors for OData 2.0 and OData 4.0 • Trailhead Module • https://trailhead.salesforce.com/en/content/learn/modules/lightning_connect/lightn ing_connect_introduction
- 23. Mulesoft OData API Kit Extension Buildsservices from the entity data model (EDM) written in RAML Supports OData 2.0 (not 4.0) Prerequisites • OData Plugin • Mule EE 4.1.1 and later • Anypoint Studio 7.1.4 and later • Maven https://docs.mulesoft.com/apikit/4.x /creating-an-odata-api-with-apikit
- 24. Mulesoft API KitExtension • Define the API in RAML first. • Use the Alt Menu -> Mule -> Generate Flows from REST API to generate the flow skeleton. • Must set the version on the HTTP Listener rather than in the configuration otherwise the API Kit extension doesn’t initialize properly. <http:listener config-ref=”http-listener-config” path=“/v1/*”>
- 25. Translating OData Query parameters We musttransform the query parameters to make them work for the underlying service. The example from Mulesoft works for a MySQL database. The following examples are for a SOAP web service. To keep our app agnostic of the fields in the SOAP service we use XQuery to transform the payload using our select, filter, orderby, top, skip, and limit variables.
- 26. OData Query Parameters ODatauses query parameters to provide operations to our app. These are the operations we need for a read-only service. • select : Defines which columns or fields in the entity to return. • filter : Defines the ‘WHERE’ clause for the query to restrict the records returned based on criteria. • orderby : Defines the sort order of the result set. • top : Defines the number of records to return in the first page i.e. TOP 10 returns the first 10 records based on the orderby. • skip : Defines how many of the records at the top of the result set to ignore. This is used by the client to page through the result set. • limit : Defines the maximum number of records to return across all pages.
- 28. Translating Query Parameters: Select Contains a comma separated list of fields to return. A * in the select parameter indicates that all fields should be returned. For our app to work we turn the * into a list of specific fields so we call the service and use this dataweave to set a variable called select. %dw 2.0 output text/plain import * from dw::core::Objects var data = if (payload != null) payload[0] else [] var names = if (typeOf(data) as String == “Object” nameSet(data) else [] var select : String = vars.select match { case select if (select is String and select != ”” and select !=”*”) -> select as String case “*” -> joinBy(names, ”, “) as String else -> joinBy(names, “, ”) as String } default joinBy(names, ”, “) as String --- select
- 29. Translating Query Parameters: Filtering Salesforce has a few curious ways of implementing these filters. For example instead of contains it uses indexof to find a given string in a field We translate the filter operators from words into operator characters, i.e. eq -> = We translate datetimeoffset to xs:date format var odataFilterToXQFilter = (odatafilter) -> (( odatafilter replace ”eq null” with “= ‘’” replace /(?i)indexof((’*?|”*?)(w*?)(’*?|”*?),((’*?|”*?)(w*?)(’*?|”*?)))((s*?)ne(/s*?)-1)/ with (“fn:contains(“ ++ $[2] ++ “,” ++ $[4] ++ “)”) replace “ne null” with “!= ‘’” replace “ eq “ with “ = “ replace “ ne “ with “ != “ //Further replacements of operators omitted replace /(?i)(datetimeoffset’)(d{4,4}-d{1,2}-d{1,2})Td{1,2}:d{1,2}:d{1,2}(Z)(‘)/ with |( “xs:date(‘” ++ $[2] ++ $[3] ++ ”’)”) ) splitBy ” “ map (item, idx) -> ( if(names contains item ) “$tran/” ++ item else item ) ) joinBy “ “
- 30. Translating Query Parameters: Ordering The default ordering for both Salesforce and OData is ascending. The orderby parameter contains a comma separated list of fields with or without sort order indicators i.e. name asc, date descending Each ordering has two possible tokens asc, ascending and desc, descending var odataOrderToXOrder = (odataOrderBy) -> ( ( odataOrderBy replace “, ” with “ , “ replace “asc” with “ascending” replace “desc” with “descending” ) splitBy(/s/) map (item, idx) -> ( if ( names contains item ) “$tran/” ++ item else item ) ) joinBy “ “
- 31. Translating Query Parameters: XQuery The results of the previous steps are now used to construct an XQuery FLWOR expression to transform our payload. odataFilterToXQFilter is used to transform the OData filter into a form usable by XQuery. odataOrdertoXQOrder is used to transform the OData orderby parameter into a form usable by XQuery. The resulting string is stored in a variable and passed to an XML XQuery transform step. --- ‘xquery version “3.0”; declare variable $document external; <transactions> { for $tran in $document/transactions/transaction where ‘ ++ odataFilterToXQFilter(vars.filter) ++ ‘ order by ‘ ++ odataOrderToXQOrder(vars.orderby) ++ ‘ return $tran } </transactions>’
- 33. Translating Query Parameters: Selecting and Paging The previous steps have given us a payload that contains the correct full set of records for the given query parameters. The final step is to apply the paging parameters to return the correct subset of records and fields to the client. For this we use the following dataweave on our payload. %dw 2.0 output application/json var transactionCount : Number = sizeOf(payload default []) var rangeStart : Number = if(vars.skip > transactionCount) transactionCount else vars.skip var lastIndex : Number = vars.skip + (vars.top -1) var rangeEnd : Number = if(lastIndex >= transactionCount) transactionCount -1 else lastIndex --- { “entries” : if(not isEmpty(payload)) ( payload[rangeStart to rangeEnd] map ( //This filters the record set $ filterObject (( value, key) -> vars.select contains(key))) // This selects the fields to be returned ) else [] }
- 34.
- 35. Putting it alltogether • Mulesoft • Import the example project • Create a new app using the OData API Kit extensions • Look out for the version: it must be specified on the HTTP Connector directly • Implement flows for each entity and operation • Translate OData query parameters to apply to SOAP XML documents • Deploy to Anypoint Cloud Hub so it is visible to Salesforce • Salesforce External Service • Use the Service Document to create the external object • Use AtomPub not JSON as Salesforce doesn’t seem to work with JSON • Use the OData test kit to check the connection and queries • Add an indirect lookup to link the data to a parent record in Salesforce • Add the related list to the page layout
- 37. Summary Data visible inSalesforce Data stored externally Data presented in real-time
- 38. Salesforce External Object Mulesoft Banking Platform HTTPS OData Request HTTPS OData Response SOAPRequest SOAP Response Transaction List User Dataweave and XQuery magic in here
- 39.
- 40. Resources • Mulesoft • MulesoftOData API Kit documentation : https://docs.mulesoft.com/apikit/4.x/creating-an-odata-api- with-apikit • OData API Kit example project • https://docs.mulesoft.com/apikit/4.x/_attachments/apikit-odata-example-master.zip • Salesforce External Service • Trailhead Module : https://trailhead.salesforce.com/content/learn/modules/lightning_connect • Trailhead Project : https://trailhead.salesforce.com/en/content/learn/projects/quickstart-lightning- connect • External Objects Documentation : https://developer.salesforce.com/docs/atlas.en- us.object_reference.meta/object_reference/sforce_api_objects_external_objects.htm • OData • Documentation: https://www.odata.org/documentation/ • Reference Services: https://www.odata.org/odata-services/ • XQuery • XQuery 3.0 documentation : https://www.w3.org/TR/2014/REC-xquery-30-20140408/ • FLWOR : For, let, where, order by, return https://en.wikipedia.org/wiki/FLWOR
- 41.