My main reason for playing with offline AI right now is to help me get further into the Computer Science curriculum on my own. (disabled/just curious)
I have seen a few AI chat characters with highly detailed prompts that attempt to keep the LLM boxed into a cosplay character. I would like to try to create fellow students in a learning curriculum. I haven’t seen anything like this yet, but maybe someone else here has seen this or has some helpful tips. I would like to prompt a character to not directly use programming knowledge from its base tokens and only use what is available in a Lora, or a large context, or a langchain database. I would like to have the experience of learning along side someone to talk out ideas when they have the same amount of information as myself. Like, I could grab all the information for a university lecture posted online and feed it to the AI, watch and read the information myself, and work through the quizzes or question anything I do not understand with the answers restricted to my own internal context region.
Continuing from my PC, if you wanted a simulated experience watching a lecture and answering quizzes and such, it might be that watching the lecture is more than enough, especially if you have the quiz answers and test answers. Strategies like this are not new, not AI-powered, and have been decently successful without needing to pay for any courses directly.
However, if you wanted a way to ask questions to a Q&A bot while the lecture is running, you could use a combination of some sort of semantic retrieval (where you’re retrieving any relevant learning materials that are expected of you to explore as a student of the course) and providing the most recent lecture contents as context to the LLM.
For the retrieval part, I’d recommend looking at a vector database like Weaviate (potentially offline) or something like Azure Cognitive Search (online/cloud) to store snippets of the learning material - maybe sections of chapters or such - along with their embeddings (other options exist, but these are two that I’ve personally used). Note that the embeddings these databases use often come from an LLM, so for example with Weaviate, you’ll need access to something for embedding generation. Then, you’d use the question to query the database (either keyphrases, or possibly directly as is) for the relevant snippets, and have some number of those as one part of your context. You can use a transcription of the lecture to provide the second part of the context. Then finally the third part of your context could be the actual question, along with the format you want it to respond in. This way you can limit the amount of context you need to provide to the LLM (instead of needing to provide the entire set of learning materials as context).
This would be a pretty complicated project though. It’s not as simple as going on character.ai or ChatGPT and creating a carefully-crafted prompt :)
Edit: for limiting the knowledge of the LLM, this might just come down to selecting the right prompt, and even then it seems like it’d be a difficult challenge. I’m not sure that you’ll have much success here with current LLMs to be honest, but play around and see if you can get it to avoid generating answers off of materials you shouldn’t have learned yet.