Gem Learn is a tool that enhances the Google Gemini experience. Gone are the days of querying for important information and forgetting the answers hours later. With Gem Learn, flashcards can be automatically created based on the most recent query and stored in sets to be learned and most importantly - reviewed and remembered later. We also built a Pitt-specific career help chatbot trained and fine-tuned on Pitt CSC’s public resources on improving your resume, preparing for interviews, and landing your first job!
Gem Learn can be used just like Gemini (with chat history and contextual memory), but with the added functionality of being able to save query and response pairs in a simple flash card format! We noticed that the responses from Gemini are usually pretty long, so we have a custom prompt engineering pipeline that tailors the response length to be perfect for flashcards. The user can also edit the flashcard before saving it to the database, allowing for flexibility in card creation. Once the flashcards are in the database, they can be managed by placing them in user-created sets, modifying, or deleting them. We also included an analytics panel to look at categorical trends in queries to see what you tend to search for most, and a Pitt-specific career help chatbot built with retrieval augmented generation (RAG) and semantic retrieval for this busy recruiting season!
We built Gem Learn using React for our frontend, Supabase for the database, a Flask server for RAG, and the Google Gemini API. The chat tab is used to converse with Gemini just like in the actual app, the flashcards tab can be used to study your personalized flashcards, and the analytics tab can be accessed to view your chat trends. The career chatbot was built in the Python flask server backend utilizing retrieval augmentation generation (RAG), extracting data from various Pitt websites, creating vector embeddings, performing semantic retrieval, and wrapping responses with Gemini to return an overall more contextually relevant response to Pitt students' career prep queries.
SteelHacks XI 2025 Hackathon Winner: Best Pitt Inspired Demo powered by Gemini API
Edmond Niu - [email protected] Krishna Katakota - [email protected] Rahul Anantuni - [email protected] Eric Wang - [email protected]
Gem.Learn.-.SteelsHacks.2024.mp4
Gemini Chat Page
Add Flashcard
Edit Flashcard
Studying a Flashcard Set
Retrieval Augmented Generation (RAG) Career Chatbot
Flashcard Sets
Analyze Trends
Unfortunately, because of Supabase authorization restrictions, in order to use Gem Learn, Supabase has to be set up in the same format as ours. The database schema is shown below.
Once a supabase is set up as follows, a .env file has to be setup with a REACT_APP_GEMINI_API_KEY variable with a Gemini API key and a REACT_APP_SUPABASE_API_KEY variable with a supabase api key. A service_account_key.json must be setup in the flask-backend folder Link. After this, the flask-baskend should be run by using
pip install -U google-ai-generativelanguage google-auth-oauthlib google-labs-html-chunker
python app.py
Some other pip installs might be necessary. Once the flask-backend is running, open another terminal and run
npm install
npm start
to run Gem Learn! There's functionality to query Gemini, create flashcards, look at analytics, and use the Pitt career help chatbot!
