Short-Term Roadmap

[abetlen]

Update (2023-05-19)

• Imrove documentation for build / installation including available compiler options.
• Huggingface Transformers API
• Functional API and code quality improvements
• Fix performance issues related to memory allocations

---

Just wanted to get some thoughts down re: short term project goals

• Add tests or some kind of build process now that presumably more people are going to be using this project. (Add tests #24)
• Improve the packaging and release process (currently we're using setup.py which I believe is a little outdated).
• Simplify and test the implementation of stop sequences to ensure correctness in both simple and streaming modes. (Add additional tests for stop sequences #8)
• Review OpenAI compatibility of the Python API and FastAPI server (Standalone Server #21)

[MillionthOdin16]

Awesome! Until I saw this repo, I hadn't thought about using OpenAI's API spec as a standard to follow. I've seen lots of other servers that have unique implementations for their APIs, but mimicking OpenAI's standard opens up the possibility of using Llama models in tons of projects. Their use is so widespread and people understand it pretty well, so that was an awesome idea!

The the speed and quality might not be the same as GPT, but at least we can run things locally and experiment without worrying about API costs ❤️