1.
@stillinbeta // #postgresconf // NYC 2019
You got Database
in my Cloud!
Kubernetes Foreign Data Wrapper
for Postgres
Liz Frost

2.
@stillinbeta // #postgresconf // NYC 2019
Who am I?

3.
@stillinbeta // #postgresconf // NYC 2019
Who am I?

4.
@stillinbeta // #postgresconf // NYC 2019
Who am I?

5.
@stillinbeta // #postgresconf // NYC 2019
Who am I?

6.
@stillinbeta // #postgresconf // NYC 2019
Who am I?

7.
@stillinbeta // #postgresconf // NYC 2019
What is Kubernetes?
● Container Manager
● API- and Object-driven
● Highly Extensible
● Monumentally popular

8.
@stillinbeta // #postgresconf // NYC 2019
Who am I?

9.
@stillinbeta // #postgresconf // NYC 2019

10.
@stillinbeta // #postgresconf // NYC 2019
...get it?
zéro un zéro
cero uno cero
零一零

11.
@stillinbeta // #postgresconf // NYC 2019
What is a Foreign Data Wrapper (FDW)?
● ODBC
● SQLite
● Cassandra
● Redis
● Hue?
● FDWs?
● Kubernetes!

12.
@stillinbeta // #postgresconf // NYC 2019

13.
@stillinbeta // #postgresconf // NYC 2019

14.
@stillinbeta // #postgresconf // NYC 2019
void
BeginForeignScan (ForeignScanState *node,
int eflags);
TupleTableSlot *
IterateForeignScan (ForeignScanState *node);
ForeignScan *
GetForeignPlan (PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
ForeignPath *best_path,
List *tlist,
List *scan_clauses,
Plan *outer_plan); That doesn’t look like Go...

15.
@stillinbeta // #postgresconf // NYC 2019
How about Multicorn?

16.
@stillinbeta // #postgresconf // NYC 2019

17.
@stillinbeta // #postgresconf // NYC 2019

18.
@stillinbeta // #postgresconf // NYC 2019

19.
@stillinbeta // #postgresconf // NYC 2019
Why not Multicorn?

20.
@stillinbeta // #postgresconf // NYC 2019
Why not Multicorn?

21.
@stillinbeta // #postgresconf // NYC 2019
Native Extension?

22.
@stillinbeta // #postgresconf // NYC 2019
Nope. No C.

23.
@stillinbeta // #postgresconf // NYC 2019
Guess I’ll go

24.
@stillinbeta // #postgresconf // NYC 2019
Success! Someone did the hard part for us!

25.
@stillinbeta // #postgresconf // NYC 2019
github.com/dennwc/go-fdw

26.
@stillinbeta // #postgresconf // NYC 2019

27.
@stillinbeta // #postgresconf // NYC 2019

28.
@stillinbeta // #postgresconf // NYC 2019
C <=> Go?

29.
@stillinbeta // #postgresconf // NYC 2019
C <=> Go?
CGo!

30.
@stillinbeta // #postgresconf // NYC 2019
// #include <stdio.h>
// #include <errno.h>
import "C"
How does cgo work?

31.
@stillinbeta // #postgresconf // NYC 2019
//#cgo CFLAGS:
-I/usr/include/postgresql/9.6/server
-I/usr/include/postgresql/internal
//#cgo LDFLAGS: -Wl,-unresolved-symbols=ignore-all
//#include "postgres.h"
//#include "funcapi.h"
<snip>
Import “C”
How does cgo work?

32.
@stillinbeta // #postgresconf // NYC 2019
// Property adds a key-value property to results of EXPLAIN query.
func (e Explainer) Property(k, v string) {
C.ExplainPropertyText(C.CString(k), C.CString(v), e.es)
}
So how do we call?

33.
@stillinbeta // #postgresconf // NYC 2019
//static Datum cStringGetDatum(const char *str) {
// PG_RETURN_TEXT_P(CStringGetTextDatum(str));
//}
How do we use macros?

34.
@stillinbeta // #postgresconf // NYC 2019
type Table interface {
Stats(opts *Options) TableStats
Scan(rel *Relation, opts *Options) Iterator
}
type Iterator interface {
Next() []interface{}
Reset()
Close() error
}
What’s our interface look like?

35.
@stillinbeta // #postgresconf // NYC 2019
type Table interface {
Stats(opts *Options) (TableStats, error)
Scan(rel *Relation, opts *Options) (Iterator, error)
}
type Iterator interface {
Next() ([]interface{}, error)
Reset()
Close() error
}
Wouldn’t be Go without errors!

36.
@stillinbeta // #postgresconf // NYC 2019
CREATE SERVER IF NOT EXISTS kind
FOREIGN DATA WRAPPER k8s_fdw
OPTIONS (kubeconfig '/kubeconfig');
What kind of interface do we want?

37.
@stillinbeta // #postgresconf // NYC 2019
What’s a Kubeconfig?

38.
@stillinbeta // #postgresconf // NYC 2019
apiVersion: v1
clusters:
- cluster:
server: http://localhost:8080
name: local-server
contexts:
- context:
cluster: local-server
namespace: the-right-prefix
user: myself
name: default-context
current-context: default-context
kind: Config
preferences: {}
users:
- name: myself
user:
password: secret
username: admin
Connection
Credentials

39.
@stillinbeta // #postgresconf // NYC 2019
What does a Kubernetes
object look like?

40.
@stillinbeta // #postgresconf // NYC 2019
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['sh', '-c', 'echo Hello Kubernetes!
&& sleep 3600']
- name: myapp-sidecar
image: stillinbeta/logging
status:
conditions:
- type: Ready
status: "True"
lastProbeTime: null
lastTransitionTime: 2018-01-01T00:00:00Z
Many fields, variable lengths, hierarchical...

41.
@stillinbeta // #postgresconf // NYC 2019
11 fields
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['sh', '-c', 'echo Hello Kubernetes!
&& sleep 3600']
- name: myapp-sidecar
image: stillinbeta/logging
status:
conditions:
- type: Ready
status: "True"
lastProbeTime: null
lastTransitionTime: 2018-01-01T00:00:00Z

42.
@stillinbeta // #postgresconf // NYC 2019
11 fields
...and this is a
small object
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['sh', '-c', 'echo Hello Kubernetes!
&& sleep 3600']
- name: myapp-sidecar
image: stillinbeta/logging
status:
conditions:
- type: Ready
status: "True"
lastProbeTime: null
lastTransitionTime: 2018-01-01T00:00:00Z

43.
@stillinbeta // #postgresconf // NYC 2019
apiVersion: v1
kind: Pod
metadata:
annotations:
kubernetes.io/config.hash: 4b52d75cab61380f07c0c5a69fb371d4
kubernetes.io/config.mirror: 4b52d75cab61380f07c0c5a69fb371d4
kubernetes.io/config.seen: "2019-03-19T16:11:31.520506966Z"
kubernetes.io/config.source: file
scheduler.alpha.kubernetes.io/critical-pod: ""
creationTimestamp: "2019-03-19T16:12:57Z"
labels:
component: kube-scheduler
tier: control-plane
name: kube-scheduler-kate
namespace: kube-system
resourceVersion: "446"
selfLink: /api/v1/namespaces/kube-system/pods/kube-scheduler-kate
uid: dc176fb5-4a61-11e9-8e22-080027245760
spec:
containers:
- command:
- kube-scheduler
- --address=127.0.0.1
- --kubeconfig=/etc/kubernetes/scheduler.conf
- --leader-elect=true
image: k8s.gcr.io/kube-scheduler:v1.13.4
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 8
httpGet:
host: 127.0.0.1
path: /healthz
port: 10251
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 15
name: kube-scheduler
resources:
requests:
cpu: 100m
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/kubernetes/scheduler.conf
name: kubeconfig
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
hostNetwork: true
nodeName: kate
priority: 2000000000
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
operator: Exists
volumes:
- hostPath:
path: /etc/kubernetes/scheduler.conf
type: FileOrCreate
name: kubeconfig
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2019-03-19T16:11:32Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2019-03-19T16:11:36Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2019-03-19T16:11:36Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2019-03-19T16:11:32Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://aee2c6b8ed3a23a636f235183fbadaa6e827660ab49a7d8742a73f5143d376fd
image: k8s.gcr.io/kube-scheduler:v1.13.4
imageID: docker-pullable://k8s.gcr.io/kube-scheduler@sha256:e7b2f9b1dcfa03b0e43b891979075d62086fe14e169de081f9c23db378f5b2f7
lastState: {}
name: kube-scheduler
ready: true
restartCount: 0
state:
running:
startedAt: "2019-03-19T16:11:34Z"
hostIP: 10.0.2.15
phase: Running
podIP: 10.0.2.15
qosClass: Burstable
startTime: "2019-03-19T16:11:32Z"
Here’s a real-world
example

44.
@stillinbeta // #postgresconf // NYC 2019
We probably don’t
care about every field!

45.
@stillinbeta // #postgresconf // NYC 2019
CREATE SERVER IF NOT EXISTS kind
FOREIGN DATA WRAPPER k8s_fdw
OPTIONS (kubeconfig '/kubeconfig');
CREATE FOREIGN TABLE IF NOT EXISTS pods (
name text OPTIONS (alias 'metadata.name')
, namespace text OPTIONS (alias 'metadata.namespace')
)
SERVER kind
OPTIONS (
namespace 'kube-system'
, apiVersion 'v1'
, kind 'Pod'
);
What kind of interface do we want?

46.
@stillinbeta // #postgresconf // NYC 2019
$ psql --user postgres < test.sql
CREATE EXTENSION
CREATE SERVER
CREATE FOREIGN TABLE
Let’s try it out!

47.
@stillinbeta // #postgresconf // NYC 2019
$ psql --user postgres < test.sql
CREATE EXTENSION
CREATE SERVER
CREATE FOREIGN TABLE
ERROR: couldn't get kubeconfig: couldn't get resource mapper:
could not get api group resources: Get
https://ec2-54-205-250-176.compute-1.amazonaws.com:6443/api?ti
meout=32s: net/http: invalid header field value "postgres:
postgres postgres [local]
SELECTx00x00x00x00x00x00x00x00x00x00/v0.0.0
(linux/amd64) kubernetes/$Format" for key User-Agent
oh.

48.
@stillinbeta // #postgresconf // NYC 2019
$ psql --user postgres < test.sql
CREATE EXTENSION
CREATE SERVER
CREATE FOREIGN TABLE
name | namespace
--------------------------------+-------------
coredns-6f685fffbf-7xrtp | kube-system
coredns-6f685fffbf-nzfn7 | kube-system
etcd-master | kube-system
kube-apiserver-master | kube-system
kube-controller-manager-master | kube-system
kube-proxy-lk2mq | kube-system
kube-scheduler-master | kube-system
That’s better!

49.
@stillinbeta // #postgresconf // NYC 2019
That’s boring. What
else can you do?

50.
@stillinbeta // #postgresconf // NYC 2019
How about
JSONPATH?

51.
@stillinbeta // #postgresconf // NYC 2019
...what’s JSONPATH?

52.
@stillinbeta // #postgresconf // NYC 2019
text the plain text kind is {.kind} kind is List
@ the current object {@} the same as input
. or [] child operator {.kind} or {[‘kind’]} List
.. recursive descent {..name} 127.0.0.1 127.0.0.2 myself e2e
* wildcard. Get all objects {.items[*].metadata.name} [127.0.0.1 127.0.0.2]
[start:end :step] subscript operator {.users[0].name} myself
[,] union operator {.items[*][‘metadata.name’,
‘status.capacity’]}
127.0.0.1 127.0.0.2 map[cpu:4]
map[cpu:8]
?() filter {.users[?(@.name==“e2e”)].user.
password}
secret
range, end iterate list {range
.items[*]}[{.metadata.name},
{.status.capacity}] {end}
[127.0.0.1, map[cpu:4]]
[127.0.0.2, map[cpu:8]]
“ quote interpreted string {range
.items[*]}{.metadata.name}{’t’}{
end}
127.0.0.1 127.0.0.2

53.
@stillinbeta // #postgresconf // NYC 2019
text the plain text {.kind}
. or [] child operator {.kind} or {[‘kind’]}
[start:end :step] subscript operator {.users[0].name}

54.
@stillinbeta // #postgresconf // NYC 2019
Why JSONPath?

55.
@stillinbeta // #postgresconf // NYC 2019
Why JSONPath?
$ kubectl get pods -o=jsonpath='{@}'
$ kubectl get pods -o=jsonpath='{.items[0]}'
$ kubectl get pods -o=jsonpath='{.items[0].metadata.name}'
$ kubectl get pods -o=jsonpath='{range.items[*]}{.metadata.name}{"t"}{.status.startTime}{"n"}{end}'

56.
@stillinbeta // #postgresconf // NYC 2019
CREATE FOREIGN TABLE IF NOT EXISTS pods (
name text OPTIONS (alias 'metadata.name')
, namespace text OPTIONS (alias 'metadata.namespace')
, container text OPTIONS (alias '{.spec.containers[0].image}')
)
SERVER kind
OPTIONS (
namespace 'kube-system'
, apiVersion 'v1'
, kind 'Pod'
);
Let’s do that!

57.
@stillinbeta // #postgresconf // NYC 2019
SELECT * from PODS;
name | namespace | container
--------------------------------+-------------+--------------------------------------------
coredns-6f685fffbf-7xrtp | kube-system | k8s.gcr.io/coredns:1.2.6
coredns-6f685fffbf-nzfn7 | kube-system | k8s.gcr.io/coredns:1.2.6
etcd-master | kube-system | k8s.gcr.io/etcd:3.2.24
kube-apiserver-master | kube-system | k8s.gcr.io/kube-apiserver:v1.12.1
kube-controller-manager-master | kube-system | k8s.gcr.io/kube-controller-manager:v1.12.1
kube-proxy-lk2mq | kube-system | k8s.gcr.io/kube-proxy:v1.12.1
kube-scheduler-master | kube-system | k8s.gcr.io/kube-scheduler:v1.12.1

58.
@stillinbeta // #postgresconf // NYC 2019
That’s fine and dandy,
but not everything’s
a string!

59.
@stillinbeta // #postgresconf // NYC 2019
Labels
object
Map of string keys and values that can be used to organize and
categorize (scope and select) objects. May match selectors of replication
controllers and services.

60.
@stillinbeta // #postgresconf // NYC 2019
How about a map?

61.
@stillinbeta // #postgresconf // NYC 2019
CREATE FOREIGN TABLE IF NOT EXISTS pods (
name text OPTIONS (alias 'metadata.name')
, namespace text OPTIONS (alias 'metadata.namespace')
, container text OPTIONS (alias '{.spec.containers[0].image}')
, labels jsonb OPTIONS (alias 'metadata.labels')
)
SERVER kind
OPTIONS (
namespace 'kube-system'
, apiVersion 'v1'
, kind 'Pod'
);
What kid of interface do we want?

62.
@stillinbeta // #postgresconf // NYC 2019
SELECT name, labels FROM pods;
name | labels
--------------------------------+-----------------------------------------------------------------------------------------------------
coredns-6f685fffbf-7xrtp | {"k8s-app": "kube-dns", "pod-template-hash": "6f685fffbf"}
coredns-6f685fffbf-nzfn7 | {"k8s-app": "kube-dns", "pod-template-hash": "6f685fffbf"}
etcd-master | {"tier": "control-plane", "component": "etcd"}
kube-apiserver-master | {"tier": "control-plane", "component": "kube-apiserver"}
kube-controller-manager-master | {"tier": "control-plane", "component": "kube-controller-manager"}
kube-proxy-lk2mq | {"k8s-app": "kube-proxy", "pod-template-generation": "1", "controller-revision-hash": "6cbfff58bb"}
kube-scheduler-master | {"tier": "control-plane", "component": "kube-scheduler"}
(7 rows)

63.
@stillinbeta // #postgresconf // NYC 2019
SELECT name, container, labels->'component' AS component FROM pods;
name | container | component
--------------------------------+--------------------------------------------+---------------------------
coredns-6f685fffbf-7xrtp | k8s.gcr.io/coredns:1.2.6 |
coredns-6f685fffbf-nzfn7 | k8s.gcr.io/coredns:1.2.6 |
etcd-master | k8s.gcr.io/etcd:3.2.24 | "etcd"
kube-apiserver-master | k8s.gcr.io/kube-apiserver:v1.12.1 | "kube-apiserver"
kube-controller-manager-master | k8s.gcr.io/kube-controller-manager:v1.12.1 | "kube-controller-manager"
kube-proxy-lk2mq | k8s.gcr.io/kube-proxy:v1.12.1 |
kube-scheduler-master | k8s.gcr.io/kube-scheduler:v1.12.1 | "kube-scheduler"

64.
@stillinbeta // #postgresconf // NYC 2019
And for my
final trick:

65.
@stillinbeta // #postgresconf // NYC 2019
Pod Pod Pod PodPod

66.
@stillinbeta // #postgresconf // NYC 2019
Deployment
ReplicaSet ReplicaSet ReplicaSet
Pod Pod Pod Pod Pod Pod

67.
@stillinbeta // #postgresconf // NYC 2019
coredns
coredns-6fec coredns-adc1 coredns-14ac
coredns
-6fec
-add7
coredns
-6fec
-7ae4
coredns-
adc1
-bbd5
coredns-
adc1
-1aed
coredns-
14ac
-86ef
coredns-
14ac
-d4ec

68.
@stillinbeta // #postgresconf // NYC 2019
coredns
coredns-6fec coredns-adc1 coredns-14ac
coredns
-6fec
-add7
coredns
-6fec
-7ae4
coredns-
adc1
-bbd5
coredns-
adc1
-1aed
coredns-
14ac
-86ef
coredns-
14ac
-d4ec
deployments
replica_sets
pods

69.
@stillinbeta // #postgresconf // NYC 2019
SELECT "deployments"."name" AS deployment_name
, "replica_sets"."name" as replica_name
, "pods"."name" AS pod_name
FROM deployments
JOIN replica_sets on "replica_sets"."name" LIKE "deployments"."name" || '-%'
JOIN pods on "pods"."name" LIKE "replica_sets"."name" || '-%';
deployment_name | replica_name | pod_name
-----------------+--------------------+--------------------------
coredns | coredns-6f685fffbf | coredns-6f685fffbf-7xrtp
coredns | coredns-6f685fffbf | coredns-6f685fffbf-nzfn7

70.
@stillinbeta // #postgresconf // NYC 2019
● Column types mostly ignored
What doesn’t work?

71.
@stillinbeta // #postgresconf // NYC 2019
● Column types mostly ignored
● If it’s a not a number, string, or map...
...just kind of give up
What doesn’t work?

72.
@stillinbeta // #postgresconf // NYC 2019
● Column types mostly ignored
● If it’s a not a number, string, or map...
...just kind of give up
● The codebase not being a knot of
spaghetti
What doesn’t work?

73.
@stillinbeta // #postgresconf // NYC 2019
● Column types mostly ignored
● If it’s a not a number, string, or map...
...just kind of give up
● The codebase not being a knot of
spaghetti
● Read-only
What doesn’t work?

74.
@stillinbeta // #postgresconf // NYC 2019
Lessons from
Production

75.
@stillinbeta // #postgresconf // NYC 2019
Lessons from
Production
The week I wrote this

76.
@stillinbeta // #postgresconf // NYC 2019
Debugging is Hard!
● Lots of Segfaults
● Postgres codebase large and intimidating
● Go Problem? C Problem? Both?

77.
@stillinbeta // #postgresconf // NYC 2019
PGXS?
● Makefile based
● C Only :(

78.
@stillinbeta // #postgresconf // NYC 2019
“For more
complicated
packages, you might
need to write your
own build system.”

79.
@stillinbeta // #postgresconf // NYC 2019
Bazel + Docker
● Bazel builds C + Go
● Docker container runs postgres
bazel run //:k8s_fdw_image
docker run
-v /tmp/config:/kubeconfig
--rm --name=k8s_fdw
bazel:k8s_fdw_image

80.
@stillinbeta // #postgresconf // NYC 2019
Questions?
Concerns?
Come find me! I’m the one with
pink hair!
Twitter: @stillinbeta
Github: github.com/liztio/k8s-fdw
K8s.slack.io: @liz