Featurebyte focuses on accelerating data science through a feature platform that facilitates feature engineering, with an emphasis on transactional data and generative AI. The document discusses the challenges of feature engineering, the benefits of transformers, and how generative AI can assist in context-aware feature ideation. Innovations in feature platforms are presented as solutions to common pain points, enhancing production accuracy and fostering creativity in data science.

1. FeatureByte
Accelerating Data Science
through Feature Platform,
Transformers and GenAI
Sept 2023
Xavier Conort
with the help of Google Bard, ChatGPT and FeatureByte!

2. FeatureByte
Agenda
2
● Define feature engineering
● The features I like to extract from transactional data
● Why feature engineering can be hard
● Pain points solved by Feature Platforms
● The magic of Transformers
● How Generative AI helped FeatureByte build a context-aware
automated feature ideation

3. FeatureByte
A little about me
3
Actuary
Learnt to formulate
mathematically
insurance problems and
solved them with data.
Kaggle addict
Learnt to have fun with
data.
Chief Data Scientist,
DataRobot
Learnt to codify best practices.
Co-founder,
FeatureByte
Solving remaining pain points
and planning to have fun with
data again!

4. FeatureByte
Define feature
engineering

5. FeatureByte 5

6. FeatureByte
2 types of features
6
1. Transformed features:
a. combine columns, binning…
b. model specific: scaling, one-hot encoding, embedding, reducing, …
2. Features from raw data:
a. From regular time series
b. From transactional data
c. From slowly changing dimension tables
Well covered by books and blogs, well supported by
tools like sklearn and automated by Auto-ML
Pretty well covered by books and blogs and
automated by Auto-ML, TS package or deep learning
The most exciting but challenging feature
engineering task.

7. FeatureByte
The features I like to
extract from
transactional data

8. FeatureByte 8
across a variety of signal types and entities associated with the use case
1. Attribute: gets the attribute of the entity at a point-in-time
2. Frequency: counts the occurrence of events
3. Recency: measures the time since the latest event
4. Timing: relates to when the events happened
5. Latest event: attributes of the latest event
6. Stats: aggregates a numeric column's values
7. Diversity: measures the variability of data values
8. Stability: compares recent events to those of earlier periods
9. Similarity: compares an individual entity feature to a group
10. Most frequent: gets the most frequent value of a categorical column
11. Bucketing: aggregates a column's values across categories of a categorical column.
12. Attribute stats: collects stats for an attribute of the entity
13. Attribute change: measures the occurrence or magnitude of changes to slowly changing attributes
14. Location…
The MEANINGFUL ones!

9. FeatureByte 9
Dimension table with
static description on
products
Item table with details on
purchased products in
customer invoices
Event table
where each
row indicates
an invoice in a
Grocery shop
Slowly Changing Dimension table that contains
data on customers that change over time.
Simple example of transaction data: A Grocery Dataset

10. FeatureByte 10
Invoice x Similarity
Customer x Diversity
Customer x Attribute change
Customer x Similarity

11. FeatureByte 11
Customer x Timing
Features inspired by Owen Zhang, Kaggle
Grandmaster

12. FeatureByte 12
More than 100 examples in the tutorials of FeatureByte’s free and
source available package

13. FeatureByte
Why feature
engineering can be hard

14. FeatureByte
Feature Engineering with transactional data is Complex and Challenging
14
1. Transactional data can be
XXXL
2. Feature space can be
infinite
3. High risk of time leakage
and training-serving
inconsistencies: need to be
point-in-time accurate
4. Feature engineering
strategy varies with the
data semantics
5. Garbage-in Garbage-out

15. FeatureByte
3 skills come into play
15
DOMAIN
EXPERTISE
DATA
ENGINEERING
DATA
SCIENCE
DW
SAAS
SOURCES
MODEL
TRAINING
MODEL
PREDICTION
STREAMING

16. FeatureByte
… while 3 innovations are already radically simplifying the effort
16
DOMAIN
EXPERTISE
DATA
ENGINEERING
DATA
SCIENCE
Generative AI is
getting close to
emulate a
Domain Expert
Feature
Platforms are
simplifying data
engineering
Off the Shelf
Transformers are
revolutionizing NLP

17. FeatureByte
Pain points solved by
feature platforms

18. FeatureByte 18
Pain Point How a Feature Platform is addressing it
High latency in feature computation Pre-computes features and stores them in
an online store for low latency
Disappointing production accuracy caused
by inconsistency of feature values between
training and production
Ensures point-in-time correctness and
training-serving consistency
Code for efficient time travel operations in
SQL is complex
Declarative framework in Python to simplify
the creation of features
Waiting to experiment with our new feature
ideas can be frustrating
Automated backfilling to compute features at
any point in time in the past
Painful search for existing features Feature Catalog to share and reuse features
Uncertainties when our new features will be
finally deployed
Unified solution to ensure smooth transition
from experimentation to deployment

19. FeatureByte
The magic of
Transformers

20. FeatureByte 20

21. FeatureByte
What do I think about Transformers?
21
1. As a scientist: they are beautiful.
2. As a data scientist: they are awesome.
3. As a MLOps engineer: another transformation to operationalize.
4. As a product manager: they are opening up new horizons!

22. FeatureByte
Immediate benefits of Transformers
22
1. Radically simplify a complex task (NLP)
2. In many cases, don’t need any training for great outcomes! Can be used off
the shelf to transform raw text columns of transactional data:
a. Text embedding,
b. Name Entity Recognition,
c. Sentiment analysis,
d. Keyword extraction,
e. Extract specific information in a table structure…
3. I just need to aggregate the outputs in a meaningful way

23. FeatureByte
Do I still need Feature Platforms for Transformers?
23
Feature Platforms can improve Transformers operationalization
1. Point-in-Time correctness
2. Low latency thanks to the pre-computation of features that involve
transformers
3. Caching (via partial aggregations or other mechanisms) to reduce expensive
transformer calls
4. Transformer Library

24. FeatureByte
Can Deep Learning also replace traditional feature engineering?
24
Maybe in the future, but less magic expected…
1. I have not seen anything convincing yet apart from:
a. regular time series
b. sequences of event (successful results for recommendation system)
2. If solutions emerge, it won’t be pre-trained Off the Shelf Transformers like
for NLP and you will need a lot of data to train them to get meaningful
results.
3. And for many of us, it may not pass the model validation process:
a. not explainable enough
b. and maybe not robust enough if your data is not XXXL

25. FeatureByte 25
"Semantics is to artificial intelligence as physics is
to engineering."
John McCarthy, considered to be one of the most important figures in the
history of artificial intelligence
"Creativity is just connecting things."
Steve Jobs
So why do I think Transformers are revolutionary?

26. FeatureByte
How Generative AI can
help feature
engineering

27. FeatureByte
Areas where Generative AI is already doing great
27
Generative AI:
1. is already very familiar with data modeling concepts
2. can recognize the semantic of the data columns well beyond numeric and
string if meaningful names or good descriptions are provided
3. already possesses deep domain knowledge that can help
a. take some important decisions in the feature engineering process
b. assess how relevant a feature is to a use case

28. FeatureByte
How FeatureByte’s Copilot is currently leveraging GenAI
28
Description of Data
(tables, columns, entities)
Data Semantics &
Domain Insights
Feature Ideation
Engine
Feature code &
descriptions
Relevance Score
and Explanation
GenAI GenAI
Context-aware
Feature suggestions
Description of Use Case
(domain, context, target)
that are interpretable and
relevant! And editable.

29. FeatureByte 29

30. FeatureByte
Generative AI is already
familiar with data
modeling concepts

31. FeatureByte
Knows best practices
31

32. FeatureByte
Knows what is an entity which is a key concept in feature engineering
32

33. FeatureByte
Generative AI can
recognize data columns
semantics

34. FeatureByte
Make the difference between different types of numeric columns
34

35. FeatureByte
Generative AI can help
design your feature
engineering strategy

36. FeatureByte
Example: Can recommend strategy to filter data
36

37. FeatureByte
Generative AI can
assess the feature
relevance to a use case

38. FeatureByte
Plain English explanation
38

39. FeatureByte
… even for complex features
39

40. FeatureByte
… and is sharing its domain knowledge during the process
40

41. FeatureByte
Conclusion

42. FeatureByte
The 3 innovations are accelerating Data Science by
42
1. Reducing time-to-value thanks to:
a. quicker experimentation, and smooth transition to deployment
2. Increasing production accuracy thanks to:
a. training / serving consistency
b. transformers
c. context aware feature generation
3. Increasing Transparency thanks to:
a. Plain English descriptions of features
b. Relevance explanations to a use case

43. FeatureByte
and are opening up new horizons
43
New opportunities to:
● bridge the gap between features and non-technical stakeholders and
contribute to better Model Explainability and stronger regulation
● boost your creativity and domain knowledge
● build solutions and not only models!
● and more time to explore other advanced solutions like causal modelling +
A/B testing that may be better addressing your use cases

44. FeatureByte
Thanks!
If you are curious to know more about Featurebyte
White Paper on FeatureByte Copilot
Video of FeatureByte Enterprise
Github Repo of the free and source available engine: https://github.com/featurebyte/featurebyte
Documentation: https://docs.featurebyte.com/0.5/