The document provides wireframes and workflows for a CCS DDS UI. It includes screens and flows for makers to create views from data sources, add metadata, upload Python scripts, validate data, and send views to checkers. It also includes screens and flows for checkers to get view data, promote views between environments, and schedule view deployments. It discusses challenges with real-time/near real-time data and notes that manual tasks include uploading new source/attribute metadata and validating view data. Validation and maintenance tasks would require SQL, Python, Git, and BigTable skills from resources.
In this session Satoru Goto, Solutions Engineer at MariaDB, shows how the Pentaho connector for MariaDB ColumnStore can be used for both BI/reporting on MariaDB ColumnStore as well as loading data into MariaDB ColumnStore.
進擊的 ASP.NET Web API 2 巨人 – 打造支援各種裝置及平台的服務
你/妳知道WebApi (REST)己經成為一個企業IT系統整合及網路服務成長最為快速的趨勢嗎? 你/妳知道現今在網路上最被Mobile device使所用的資料交換的格式是JSON (Javascript Object Notation)而不再是肥大擁腫的XML怪獸嗎?
全世界的網路大咖都拼命地開發WebApi的服務來吸引Mobile或Web application的開發者, 我們將在這堂課中結合AngularJs與ASP.NET Web API 2來讓大家開始感受WebApi的吸引力。
Database continuous integration, unit test and functional testHarry Zheng
Discuss continuous integration for database projects, including building project, deploying project to database, and executing unit tests and functional tests.
This presentation will also discuss database test standards, tips and tricks.
In this session Satoru Goto, Solutions Engineer at MariaDB, shows how the Pentaho connector for MariaDB ColumnStore can be used for both BI/reporting on MariaDB ColumnStore as well as loading data into MariaDB ColumnStore.
進擊的 ASP.NET Web API 2 巨人 – 打造支援各種裝置及平台的服務
你/妳知道WebApi (REST)己經成為一個企業IT系統整合及網路服務成長最為快速的趨勢嗎? 你/妳知道現今在網路上最被Mobile device使所用的資料交換的格式是JSON (Javascript Object Notation)而不再是肥大擁腫的XML怪獸嗎?
全世界的網路大咖都拼命地開發WebApi的服務來吸引Mobile或Web application的開發者, 我們將在這堂課中結合AngularJs與ASP.NET Web API 2來讓大家開始感受WebApi的吸引力。
Database continuous integration, unit test and functional testHarry Zheng
Discuss continuous integration for database projects, including building project, deploying project to database, and executing unit tests and functional tests.
This presentation will also discuss database test standards, tips and tricks.
Embrace The Latest Tableau Innovations Right From Day One
Want to get the most out of the latest version of Tableau? Curious to know what are the new innovations? Well, it’s time to upgrade your Tableau deployment!
- The new innovations available in the latest Tableau release (data prep & data model, visualization, IT governance)
- Tableau upgrade methodology
- How to relieve the pain of testing thanks to Kinesis-CI.
Our partner: https://systechusa.com/
Slide deck from BrightGen's Spring '23 Salesforce release webinar. Presented by Keir Bowden, CTO, and covering the key new features that are of interest to our clients. You can find the recording on our youtube channel at : https://www.youtube.com/watch?v=ONjv5BvXccY
This presentation is row of demos that introduce how to use Application Insights, how it works and how to build your own application telemetry on top of it. Two surprise demos show audience some case studies how to use Application Insights to plan hosting of global web site and how to support sales and logistics departments in real-time.
Wouldn’t it be good to know how your application or service is being used and is performing while its running live? It is essential to have more insights into the running application as the cycle time for delivering new features and releases speed up. We will cover how to detect, triage and diagnose different scenarios and provide the necessary input to quickly and correctly act to resolve situations. The focus is on web applications or services running on-premise or hosted in the cloud.
Why Standards-Based Drivers Offer Better API IntegrationNordic APIs
As enterprises grow, so too does the number and variety of data sources they use to drive business. The average company is using at least sixteen SaaS applications and has data in at least that many on-premises data stores and internal apps.
With such disparate data, each tied to a unique API, integrating, managing, and maintaining integrations for all of a company’s data creates a whole new set of challenges. Thankfully, solutions exist that enable enterprises to rely on data to drive business without causing undue strain. In this session, we’ll explore and compare the different options for solving the data integration problem and explain why you should be using standards-based drivers to abstract your API integrations.
Why Standards-Based Drivers Offer Better API IntegrationJerod Johnson
A brief overview of API integration solutions (direct, SDK, middleware, drivers) and an argument in favor of using drivers to solve your integration needs.
Did you know that just a handful of root causes cause the majority of application issues like crashes, slow performance or incorrect application behavior? Non-optimized database access, deployment mistakes, memory leaks, or inefficient coding are just some examples. Companies that think Continuous Delivery and DevOps will solve all their problems typically fail as they just run into these problems faster. In this session we take a closer look at the most common problems, how to detect them and how to incorporate performance into your DevOps culture by automatically detecting these top problems.
Process Analytics with Oracle BPM Suite 12c and BAM - OGh SIG SOA & BPM, 1st ...Lucas Jellema
Business Processes implemented in BPEL and BPM(N) and running on Oracle BPM Suite 12c or SOA Suite 12c have to fulfill a business purpose and as such must meet business requirements - both functionally and non-functionally. SLAs for throughput, response time, quality are usually associated with these processes and we typically also would like insight in the number of process executions (per group) and the paths taken through our processes.
This presentation introduces process analytics in both BPEL and BPM processes in Oracle SOA Suite and BPM Suite 12c. It explains how to configure out of the box generic analytics and process specific business indicators. The presentation than introduces BAM 12c. It demonstrates the out of the box process analytics reports and dashboards. Then it explains how to create custom reports on the unified process analytics star schema or on custom tables. Finally the presentation goes into real-time monitoring in BAM using JMS and enterprise message resources in combination with the event processing templates in BAM.
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
Embrace The Latest Tableau Innovations Right From Day One
Want to get the most out of the latest version of Tableau? Curious to know what are the new innovations? Well, it’s time to upgrade your Tableau deployment!
- The new innovations available in the latest Tableau release (data prep & data model, visualization, IT governance)
- Tableau upgrade methodology
- How to relieve the pain of testing thanks to Kinesis-CI.
Our partner: https://systechusa.com/
Slide deck from BrightGen's Spring '23 Salesforce release webinar. Presented by Keir Bowden, CTO, and covering the key new features that are of interest to our clients. You can find the recording on our youtube channel at : https://www.youtube.com/watch?v=ONjv5BvXccY
This presentation is row of demos that introduce how to use Application Insights, how it works and how to build your own application telemetry on top of it. Two surprise demos show audience some case studies how to use Application Insights to plan hosting of global web site and how to support sales and logistics departments in real-time.
Wouldn’t it be good to know how your application or service is being used and is performing while its running live? It is essential to have more insights into the running application as the cycle time for delivering new features and releases speed up. We will cover how to detect, triage and diagnose different scenarios and provide the necessary input to quickly and correctly act to resolve situations. The focus is on web applications or services running on-premise or hosted in the cloud.
Why Standards-Based Drivers Offer Better API IntegrationNordic APIs
As enterprises grow, so too does the number and variety of data sources they use to drive business. The average company is using at least sixteen SaaS applications and has data in at least that many on-premises data stores and internal apps.
With such disparate data, each tied to a unique API, integrating, managing, and maintaining integrations for all of a company’s data creates a whole new set of challenges. Thankfully, solutions exist that enable enterprises to rely on data to drive business without causing undue strain. In this session, we’ll explore and compare the different options for solving the data integration problem and explain why you should be using standards-based drivers to abstract your API integrations.
Why Standards-Based Drivers Offer Better API IntegrationJerod Johnson
A brief overview of API integration solutions (direct, SDK, middleware, drivers) and an argument in favor of using drivers to solve your integration needs.
Did you know that just a handful of root causes cause the majority of application issues like crashes, slow performance or incorrect application behavior? Non-optimized database access, deployment mistakes, memory leaks, or inefficient coding are just some examples. Companies that think Continuous Delivery and DevOps will solve all their problems typically fail as they just run into these problems faster. In this session we take a closer look at the most common problems, how to detect them and how to incorporate performance into your DevOps culture by automatically detecting these top problems.
Process Analytics with Oracle BPM Suite 12c and BAM - OGh SIG SOA & BPM, 1st ...Lucas Jellema
Business Processes implemented in BPEL and BPM(N) and running on Oracle BPM Suite 12c or SOA Suite 12c have to fulfill a business purpose and as such must meet business requirements - both functionally and non-functionally. SLAs for throughput, response time, quality are usually associated with these processes and we typically also would like insight in the number of process executions (per group) and the paths taken through our processes.
This presentation introduces process analytics in both BPEL and BPM processes in Oracle SOA Suite and BPM Suite 12c. It explains how to configure out of the box generic analytics and process specific business indicators. The presentation than introduces BAM 12c. It demonstrates the out of the box process analytics reports and dashboards. Then it explains how to create custom reports on the unified process analytics star schema or on custom tables. Finally the presentation goes into real-time monitoring in BAM using JMS and enterprise message resources in combination with the event processing templates in BAM.
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
This 7-second Brain Wave Ritual Attracts Money To You.!nirahealhty
Discover the power of a simple 7-second brain wave ritual that can attract wealth and abundance into your life. By tapping into specific brain frequencies, this technique helps you manifest financial success effortlessly. Ready to transform your financial future? Try this powerful ritual and start attracting money today!
ER(Entity Relationship) Diagram for online shopping - TAEHimani415946
https://bit.ly/3KACoyV
The ER diagram for the project is the foundation for the building of the database of the project. The properties, datatypes, and attributes are defined by the ER diagram.
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
Talk presented at Kubernetes Community Day, New York, May 2024.
Technical summary of Multi-Cluster Kubernetes Networking architectures with focus on 4 key topics.
1) Key patterns for Multi-cluster architectures
2) Architectural comparison of several OSS/ CNCF projects to address these patterns
3) Evolution trends for the APIs of these projects
4) Some design recommendations & guidelines for adopting/ deploying these solutions.
2. Agenda
• Updated UI Screens and flow according to recommended Automation
option
• Business Flow
• Workflow Automation – Environments view
• Realtime/Near Real time challenges
5. Work Queue Tabl
to display views
created by makers
All View
Accordion
Close State
6. Requirement Analysis
View Definition
Execution in progress
Script Executed
File ready to download
Validation in progress
Validation Completed
Review in progress
Life Cycle of View – By Maker Life Cycle of View – By Checker
Pending Review
Validation in progress
Validation Completed-
Ready to Promote
Promotion in progress
View Promoted
Ready to Schedule
View Creation Requirement
When the view creation is in
progress status of the view will
be ‘View Definition’
When the view creation is in
progress status of the view will
be ‘View Definition’
31. Maker Login
Maker
Landing Screen
Maker Click on
Create View
List
of created views
Click on
Create New button
Empty Form opens to
fill basic details
Maker Click on
Select Data Button
Source & Table
Accordion Opens
Maker Select
Source + Table
( Attributes pre
selected in Tables)
Maker Click Next
to add derived
attribute Metadata
Maker add
Derive attribute name
and Type
Maker Add
Derived Attribute
Metadata
Maker Select added
Attribute
Maker Click
Copy Data to Clipboard
Lightbox display
to confirm
Pre-Selected Values
Maker confirm
Pre-Selected Data
and proceed
Maker click on Upload
Pyspark script
Maker upload
Pyspark script
Through Upload Form
Maker click Next to Get
View Data
Maker select filters
to
Get View Data
Maker view status in
Work Queue for
Validation of View
Maker Click
Send View To Checker
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22
33. New View
Creation by
Maker
Validation by
Maker in
Sandbox
Validation by
Checker in
Sandbox
Promote View
from Sandbox to
UAT by Checker
Validation by
Checker in UAT
Schedule
Deployment
from UAT to Prod
Life cycle of View
54. Checker
Landing Screen
Click on
View
Click on
Get View Data
Display view details to
proceed to Get View
Data
Checker
view status in Work
Queue
Checker Download
Test Sample Data (CSV )
After the validation View
Will be ready to Promote
Checker Click on
Promote View
Checker fill promote
details to
Promote View
Checker Click Promote
View from Sandbox to
UAT
Checker Click on Get
View Data to Validate
Data in UAT
After the Successful
validation of View in UAT
Checker Click to Schedule
View
Checker Click View to
Schedule deployment
Checker Click on
schedule deployment
view from
UAT to Prod with CR No.
1 2 3 4
5 6 8
7
9 10 11 12
13 14
71. Notes
Manual tasks
• Any new sources/attributes to existing CDL tables will require a
manual upload to CDL metadata by CDL Data Ingestion Team.
• DDS control metadata table will be populated manually as a
weekly feed (TBD) and any changes to CDL control table will not
be propagated in real time until CDL has the capability to publish
Control table metadata changes.
• CDL metadata is not available as an API and hence it will be a
weekly extract (TBD) pulled from Sharepoint into DDS catalog.
Validation of CDL metadata is not in scope.
• Entire view Business Logic will be developed in Pyspark by a
business developer. The input tables for Pyspark code are staging
tables and output of Pyspark code will be message to Pubsub
topic.
• Once the job runs, the sample data can be downloaded by maker
and he/she has to validate data manually.
• Promote to Higher Non Prod environment –
1. Code merge
2. Check-in to git
• Promote to Prod Environment -
1. It needs a CR approval.
2. Check-in to git
3. Code merge
72. Assumptions
• CDL control table will be exposed to DDS and provide details on which CDL tables are DDS specific. This is required
• to populate DDS control metadata table.
• Adapter API to poll CDL control table to check CDL job completion.
• CDL control table will give us frequency at a job level. If there are multiple jobs populating CDL table at different frequencies, then
it will be pulled into DDS at a lowest frequency.
• The data from DDS will be extracted as a file and placed into NAS storage on-premise, however integration into BO is out of scope.
• DDS only source is CDL.
• Metadata such as CDL control table metadata, CDL metadata, view audit information can be placed on Github.
• Exception report can be generated in Datastudio from exception tables.
• Pyspark code needs to be scanned for vulnerabilities before uploading the code from UI.
73. Maintainence tasks
• As part of incremental loads, if a job fails, it will be retried thrice
(configurable) and on failure, xmatter alerts will be raised to
appropriate support team. The team needs to look into the issue and
resolve it.
• If transaction data is available and there is no corresponding master
data, needs to be written to exception table. These records won’t go
into Persistance layer. This needs to be handled by Pyspark code
provided by business developer.
• Debugging of any performance issues raised.
74. Workflow Automation Flow
(Environments view)
Data Pipelines
(Batch & Realtime)
delivering Views to
Persistence Layer
Design Environment
Execution Environment
9 Approve
view
Automated data pipelines &
API build for Views
(CDL > Staging > Persistence)
UAT
Data Pipelines
(Batch & Realtime)
delivering Views to
Persistence Layer
Prod
Checker
PROD
Views promoted &
scheduled
View Data sent back
7 Get View
Data
2 Create View
Maker
1 Browse
CDL Catalog
10 Promote
view (to UAT)
5 Get View
Data
6 Validate
View Data
View Data Sent back
Data Pipelines
(Batch & Realtime)
delivering Views to
Persistence Layer
Sandbox
Developer
3 Generate
PySpark
Code for
View
11 Get View
Data (UAT)
Views promoted &
scheduled
13 Promote
view (to Prod)
CR Number
4 Upload
PySpark Code
for View
8 Validate
Data
12 Validate
Data
75. Following are the skillset the resources needs to have to support the manual tasks as part of automation process
• Basic SQL skills
• Pyspark skills
• Git
• Performance tuning using Pyspark.
• Understanding of Bigtable for performance tuning as required.
• There are two options
• Business developer will be able to handle all manual tasks, maintenance tasks listed in previous slides.
• Along with business developer, there will be lean IT support team which does the manual and maintenance
tasks.
Post CG Contract – Support for Automation
76. • No RT/NRT use cases including data sources
• CDL doesn’t have any RT GCP service and RT data feed as of now
• Need to validate BigTable as a staging service
• NFRs for streaming not available
• Data Catalog needs to be made available for RT data elements
• Unclear on the type of transformations that the NRT flows would need
• Unclear how the users would like to see nested structure on screen and
selection of those nested attributes
Real/Near Real time Challenges