ChatGPT - a Conversational AI is a powerful tool that enables companies to provide exceptional customer experiences while safeguarding data privacy. Building in-house Conversional AI capabilities, such as using the tools of ColossalAI, Haystack and Coati, allow organizations to maintain full control over customer data and ensure its confidentiality.

1. How to build an in-house Chat GPT
Researched and
presented by

2. Agenda
1. Executive Summary
2. OpenAssistant
3. Haystack.deepset & Coati
4. Contact information for demo
Copyrighted by

3. Citynow Asia Inc, an IT solutions Japan-based-company, provides
comprehensive solutions for implementing in-house conversional AI,
powered by the latest generative AI models and domain knowledge from
B2B customers.
Our Conversional AI platform can support question answering features
and generate relevant content in Japanese from query information in real
time, all while keeping training/testing data safely in-house.
Executive Summary
Copyrighted by

4. OpenAssistant Approach
Copyrighted by

5. This pipeline includes four components:
1. User Interface: Where the user interact with the system by inputting
prompts and receiving responses.
2. Standardized Database: Storage for available data and knowledge base.
The data formats should be standardized beforehand.
3. Internal Search Engine: Retrieve data that are relevant to the prompts
from the user from the Standardized Database.
4. Conversional AI Module: Respond to the user’s prompt using retrieved
data from the Internal Search Engine.
OpenAssistant Approach
Copyrighted by

6. Key benefits when using OpenAssistant Approach
● Built-in models: in-house, not stored in third-party systems.
● Database security: securely stored on customer infrastructure.
● User-friendliness: customers can use the conversional AI without technical
domain knowledge.
● Model fine-tuning: platform for data validation and updating.
OpenAssistant Approach
Copyrighted by

7. Regard ColossalAI and Deepspeed Approach, Due to lack of Hardware power comparing to OpenAi (the
company behind ChatGPT), we will use Hardware Accelerator/Optimizor to help best combine the power of
our own DGX system and training step (provided by OpenAi) and Dataset (provided by Customer).
1. For Hardware support, candidate tools are (ColossalAi and DeepSpeed)
2. For System Architecture, we will build our system on 2 main components:
+ Infomation Retrieval System: This will enable the ability to matching question to closest information
store in database, and allow update database (knowledge space) regularly without the need to retrain
chatbot
+ Chatbot System: This will help to make the answer look more natural, humanlike.
Haystack.deepset & Coati Approach
Copyrighted by

8. The information retrieval system
(1) User sends question to system
(2) Python scripts embedding user question as vector
and compare to all document in system to find the
best match (evaluate by cosin score)
(3) The system returns the best match document and
best paragraph which can answer user’s question
(4) Python scripts convert all information (user
question, answer) into a format that can be fed to
the chatbot
(5) Chatbot converts all information above into a
natural response to answer user
Description
Haystack.deepset & Coati Approach
Copyrighted by

9. The information retrieval system is a tool that helps find relevant documents based on a user's query. In
this system, a Vector Database is used to store documents from the client.
Each document is converted into a set of numbers called vectors using a tool called "sentence-
transformers". This tool uses a pre-trained model called "paraphrase-multilingual-mpnet-base-v2" to
convert the text into vectors.
This approach ensures that the chatbot using this system will not have a knowledge cut problem like GPT
(which has a maximum amount of information it can store and use at a time). Additionally, the system can
easily add new documents to the database without needing to retrain the chatbot*.
*For more detail, please visit this link: https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2
Haystack.deepset & Coati Approach
Copyrighted by

10. Both the chatbot and information retrieval system can work with Japanese language data if they are
trained on Japanese language data. This means that the chatbot and the information retrieval system
will be able to understand and respond to queries in Japanese.
The pre-trained model used in the information retrieval system also supports Japanese language, so it
may not require retraining to work with Japanese data. However, further experimentation is required
to confirm this. If the system needs to support other languages, it is also possible to consult the list of
supported languages provided in the documentation of the pre-trained models.
For more detail, please visit this Link: https://www.sbert.net/docs/pretrained_models.html#multi-
lingual-models
Haystack.deepset & Coati Approach
Copyrighted by

11. Citynow Asia Inc
Website: https://citynow.jp/
Hotline: +81 3 4405 3731
Email: corp@citynow.jp
Facebook: https://www.facebook.com/citynow.asia
Citynow Asia株式会社 | Bunkyo-ku Tokyo | Facebook
Contact us today for further information
Copyrighted by

12. THANK YOU