映像やCG制作の現場において、AIの技術は様々な自動化・クリエイティブで利用され始めています。そのAI そのものは、従来のデジタル制作と異なる性質も持っており、道具としての AI を正しく理解しておくことも重要です。このセッションでは、既存のAI技術を紹介しつつ、どのようにクリエイティブの現場に取り入れ、理解していくのかをご紹介します。
映像やCG制作の現場において、AIの技術は様々な自動化・クリエイティブで利用され始めています。そのAI そのものは、従来のデジタル制作と異なる性質も持っており、道具としての AI を正しく理解しておくことも重要です。このセッションでは、既存のAI技術を紹介しつつ、どのようにクリエイティブの現場に取り入れ、理解していくのかをご紹介します。
Keynote at Developer Week Global: Cloud on Sep 15, 2021.
There are many, many resources for DevOps engineers: learning paths, guides and tutorials for using tools such as Terraform, Packer and Ansible to save time in provisioning and configuring reliable, predictable systems. This session looks at the other side of the equation: creating the plugins, modules and providers that abstract away upstream APIs for use by DevOps tools.
Director of Developer Evangelism Pat Patterson will explain how Citrix implemented DevOps tooling for its App Delivery & Security products, and how the company is working with its community to create tooling for its Virtual Apps & Desktops Service. Pat will explain the different approaches to creating tooling, trade-offs between them, and the lessons that Citrix has learned along the way. This session will NOT be death-by-PowerPoint! Come prepared for semi-colons, curly braces and monospaced text!
How Imprivata Combines External Data Sources for Business InsightsPat Patterson
Imprivata is a healthcare-focused IT security company. Using data integration tools from StreamSets, they built a pipeline to consolidate data from NetSuite, Salesforce, and Oracle in a cloud-based PostgreSQL data warehouse. Hear how their analytics team combined transaction, product, and support data in Einstein Analytics to support decision-making across their business functions: sales, marketing operations, product owners, and CxOs.
Data Integration with Apache Kafka: What, Why, HowPat Patterson
Presented at Orange County Advanced Analytics and Big Data Meetup, June 21 2019.
Apache Kafka has fast become the dominant messaging technology for the enterprise; if you're a data scientist or data engineer and you have not yet worked with Kafka, that situation will likely change soon! In this session, Pat Patterson, director of evangelism at StreamSets, explains what Kafka is, why it has disrupted the previous generation of messaging products, and how you can use open source products to build dataflow pipelines with Kafka, without writing code.
Project Ouroboros: Using StreamSets Data Collector to Help Manage the StreamS...Pat Patterson
On a typical day we see hundreds of downloads of StreamSets Data Collector, our open source data integration tool. We used to wrangle our download logs using a combination of the AWS S3 command line, sed, grep, awk and other tools, all run from a shell script (on my laptop!) once a week. This was a classic example of a brittle, hard to maintain, custom data integration. One day it dawned on me, "This is crazy, we have a tool that can do all this!". In this session, I'll explain how I built a dataflow pipeline to stream content delivery network (CDN) logs from S3 to MySQL in real-time, allowing us to gain valuable insights into our open source community. You'll also learn how we use the same techniques to not only gain insights into our community on Slack, but also build tools to better serve them.
Dealing with Drift: Building an Enterprise Data LakePat Patterson
Presented at San Diego Predictive Analytics Meetup, Dec 5, 2017
Cox Automotive comprises more than 25 companies dealing with different aspects of the car ownership lifecycle, with data as the common language they all share. The challenge for Cox was to create an efficient engine for the timely and trustworthy ingest of data capability for an unknown but large number of data assets from practically any source. Discover how their big data engineering team overcame data drift and are now populating a data lake, allowing analysts easy access to data from their subsidiary companies and producing new data assets unique to the industry.
Einstein Analytics (previously known as Wave Analytics) allows developers to not only create analytics applications, but also to create application templates that allow end-users to create their own analytics applications based on your master app. You, the developer, can define parameters and rules as part of the template, allowing the end-user to customize the app to their requirements. This Dreamforce 2017 session explains how to use Analytics Templates and the Analytics External Data API to automate the ingest of data from outside the platform, manipulating datasets and dataflows to provide a seamless experience for the user.
Efficient Schemas in Motion with Kafka and Schema RegistryPat Patterson
Apache Avro allows data to be self-describing, but carries an overhead when used with message queues such as Apache Kafka. Confluent’s open source Schema Registry integrates with Kafka to allow Avro schemas to be passed ‘by reference’, minimizing overhead, and can be used with any application that uses Avro. Learn about Schema Registry, using it with Kafka, and leveraging it in your application.
Dealing With Drift - Building an Enterprise Data LakePat Patterson
Data drift, the gradual morphing of data structure and semantics, is a fact of life in enterprise IT. New requirements force schema changes, the meaning of database columns changes over time, and infrastructure upgrades add new fields to log files. Left unchecked, drift in data sources can cause applications and dataflows to fail, with costly downtime and, in the worst case, corruption in downstream data stores.
Cox Automotive comprises more than 25 companies dealing with different aspects of the car ownership lifecycle, with data as the common language they all share. The challenge for Cox was to create an efficient engine for the timely and trustworthy ingest of data capability for an unknown but large number of data assets from practically any source. Discover how their big data engineering team overcame data drift and are now populating a data lake, allowing analysts easy access to data from their subsidiary companies and producing new data assets unique to the industry.
Building Data Pipelines with Spark and StreamSetsPat Patterson
Big data tools such as Hadoop and Spark allow you to process data at unprecedented scale, but keeping your processing engine fed can be a challenge. Metadata in upstream sources can ‘drift’ due to infrastructure, OS and application changes, causing ETL tools and hand-coded solutions to fail. StreamSets Data Collector (SDC) is an Apache 2.0 licensed open source platform for building big data ingest pipelines that allows you to design, execute and monitor robust data flows. In this session we’ll look at how SDC’s “intent-driven” approach keeps the data flowing, with a particular focus on clustered deployment with Spark and other exciting Spark integrations in the works.
Adaptive Data Cleansing with StreamSets and CassandraPat Patterson
Presented at Cassandra Summit 2016.
Cassandra is a perfect fit for consuming high volumes of time-series data directly from users, devices, and sensors. Sometimes, though, when we consume data from the real world, systematic and random errors creep in. In this session, we'll see how to use open source tools like RabbitMQ and StreamSets Data Collector with Cassandra features such as User Defined Aggregates to collect, cleanse and ingest variable quality data at scale. Discover how to combine the power of Cassandra with the flexibility of StreamSets to implement adaptive data cleansing.
Big data ingest frameworks ship with an array of connectors for common data origins and destinations, such as flat files, S3, HDFS, Kafka etc, but sometimes, you need to send data to, or receive data from a system that's not on the list. StreamSets includes template code for building your own connectors and processors; we'll walk through the process of building a simple destination that sends data to a REST web service, and show how it can be extended to target more sophisticated systems such as Salesforce Wave Analytics.
Ingest and Stream Processing - What will you choose?Pat Patterson
Pat Patterson and Ted Malaska talk about current and emerging technologies. They evaluate each and understand how they are useful in solving problems related to large scale data processing, joining and combining streams. They also talk about the various ways of achieving "at least once" and "exactly once" processing and how we can make sure that data is processed in a timely fashion.
Open Source Big Data Ingestion - Without the Heartburn!Pat Patterson
Big Data tools such as Hadoop and Spark allow you to process data at unprecedented scale, but keeping your processing engine fed can be a challenge. Upstream data sources can 'drift' due to infrastructure, OS and application changes, causing ETL tools and hand-coded solutions to fail, inducing heartburn in even the most resilient data scientist. This session will survey the big data ingestion landscape, focusing on how open source tools such as Sqoop, Flume, Nifi and StreamSets can keep the data pipeline flowing.
All Aboard the Boxcar! Going Beyond the Basics of RESTPat Patterson
One of the basic tenets of the REST paradigm is that resources are individually identified by URLs. REST APIs provide access to resources by HTTP operations such as GET and POST on those URLs. While this approach makes for a clean abstraction, it can be inefficient - an application that makes an HTTP request for each resource it wishes to act on may deliver a poor user experience, especially in a high latency environment such as mobile.
'Boxcarring' allows us to bundle multiple operations into a single HTTP call. For example, an application might POST a whole set of resource representations for creation. Resources carried in the boxcar can be independent, or form a tree of linked resources. This session will explore the boxcar approach, show an implementation in action, and discuss whether this can still be considered a RESTful approach.
Presented 9/30/15 at Integrate 2015.
OData: Universal Data Solvent or Clunky Enterprise Goo? (GlueCon 2015)Pat Patterson
Why would anyone but the most pedestrian enterprise developer be interested in a data access protocol originally designed by Microsoft, implemented in XML and handed to OASIS for standardization? The Open Data Protocol, or OData for short, has evolved into a clean, RESTful interface for CRUD operations against data services. Alongside the usual enterprise suspects such as Microsoft, Salesforce and IBM, OData has been adopted by government and non-profit agencies to open up their data and make it accessible to the public. For developers wanting to consume data, or create their own OData services, there's no shortage of open source options, from Apache Olingo in Java to node-odata and ODataCpp. Whether you're accessing customer orders in SAP or the Whitehouse visitor book, you're going to need some OData smarts.
The Open Data Protocol, or OData for short, provides a RESTful interface for CRUD operations against data services. OData services, such as Microsoft Azure, SAP, and WebSphere expose data and metadata as typed name/value pairs in JSON or XML, allowing 'off-the-shelf' data consumers to integrate with services without custom code. This session gives an overview of OData, and explains why salesforce.com selected it as a protocol to integrate with external data services.
API-Driven Relationships: Building The Trans-Internet Express of the FuturePat Patterson
Presented at IRM Summit, June 4 2014. Abstract:
Move over Thomas the Tank Engine! If developers are the train conductors of the railway, then APIs
are the bullet trains of the future. In this session, Pat Patterson will explain how identity allows API
ecosystems to flourish, enabling developers to build ever more elaborate integrations. Whether it be
ExactTarget, Minecraft, or other great services, APIs are the basis for empowering developers to build
boundary-less railways across the web. ALLLLLL ABOOOOOOOOOAAAARD!
1. Identity in the Cloud
- クラウドサービスとアイデンティティ Japan Identity & Cloud Summit, Tokyo, Japan, January 2014
Pat Patterson (パット・パターソン)
プリンシパルディベロッパーエヴァンジェリスト
@metadaddy
In The Past, Identity Was Only About Managing Employee Apps
Today, Employees Are Using Devices and Apps From Everywhere
Wall of Fire
Identity Challenges Grow Exponentially With Customers, Partners and Products
So what’s the solution?
An Identity PlatformCloud basedInteroperableMobile enabledInternet scaleSingle point of controlAvailable to enterprises large & smallCore to the application platform
InteroperabilitySAML 2.0OAuth 2.0OpenID Connect
Internet scale10B+Logins per year1.5B+ Transactions per day70%+ of customers are using OAuth500M+API calls per day
Identity Platform, identity at the core of the platform