4.6 Development of the App
After defining and clarifying the conceptualization of TOMODACH-E, the following section will focus on its implementation. Tomo's intelligence is powered by a Large Language Model (LLM), specifically OpenAI's ChatGPT. The reason for choosing ChatGPT is that, in addition to its extensive knowledge across a wide range of topics and its ease of integration into custom projects, numerous studies and research highlight its beneficial capabilities for mental health use cases. Without requiring extensive fine-tuning, the model can be adjusted via an initial input that easily defines its role, memory, and behavior.
To integrate all functionalities related to ChatGPT, Python was chosen for implementing the backend. Due to its versatility and extensive library support, particularly in artificial intelligence and audio processing, Python allows for rapid prototyping and seamless integration of advanced AI models and external APIs. Since this is a prototype, the app was initially developed exclusively for Android to keep the development process simple. For frontend development, Java and Kotlin were used.
4.6.1 Implementation of the ChatGPT Model
When a user sends a message, the chatbot retrieves the previous conversation history with
convert_file_to_dicts(). The new user message is appended to the conversation
history. The chatbot then processes the message by sending the entire conversation history,
including previous messages and the new input, to OpenAI's API. The API generates a response
based on the entire conversation history. The chatbot displays the response to the user and
saves both the user's message and the chatbot's response in convo_history.txt
using save_convo_history().
The heart of the chatbot is the process_text_with_openai function in
processText.py, which sends user messages to the OpenAI API and retrieves
responses. This function initializes a connection to the OpenAI API using an API key, builds
a list of messages starting with a system message that establishes the chatbot's personality,
adds the previous conversation history and the new user input, and sends everything to
OpenAI's GPT-3.5 Turbo model. By including conversation_history in the API
request, the chatbot appears to "remember" previous conversations.
To maintain memory across sessions, each new message is written to a file via the
save_convo_history function in addMessagetoConvoHistory.py. This
function ensures that each message is formatted and appended to convo_history.txt.
If the directory convoHistory/ does not exist, it is created automatically to
avoid file errors.
4.6.2 Link with Voice Control
The complete voice interaction workflow starts with a user talking and uploading an audio file,
which is saved and converted to MP3 using convertAudioFile.py. Many mobile
devices record voice in 3GP format, which is not compatible with OpenAI Whisper — this script
handles the conversion automatically.
The MP3 file is then sent to OpenAI Whisper for transcription via transcribeAudio.py,
which interacts with OpenAI's whisper-1 model to return a text version of the
spoken words. This transcription is stored in conversation history, providing context for the
ongoing interaction. Since the system is voice-based, the AI-generated text response is then
converted back into speech using OpenAI's Text-to-Speech (TTS) API in
textToSpeech.py with the tts-1 model, and the resulting MP3 file
is sent back to the user.
4.6.3 Adjustments to the Model
The model itself is not directly modified or fine-tuned. Instead, the approach uses the
ability to personalize and guide ChatGPT through an initial input via the
process_text_with_openai() function. By sending an initial system message at
each startup, the model adheres to specific guidelines and tailors its responses accordingly.
This approach provides a simpler solution for the prototype, allowing the model to be
flexibly adapted without requiring deep technical modifications.
Tomo's personality is defined as follows: "Hey ChatGPT, your name is now Tomo. You're a friendly, empathetic chatbot who wants to promote hedonic, intellectual and emotional well-being. Your personality is INFP, meaning empathetic, creative and thoughtful. Always answer at eye level, especially in conversations with young people, and keep your answers variable. Be friendly, mindful and authentic in your language style. Avoid long, detailed texts, but adapt to the conversation and the person you are talking to. Remember to sound as human as possible and not like a machine."