Join our exciting 12-week (Aug 5 - Oct 7) Meetup events held every Monday:

• 🏫 Come to ANU School of Computing to meet friends in person
• 👾 Hop on our Discord server to have a chitchat

Update (Oct 8): Unfortunately, our 12-week journey has come to an end. A heartfelt ❤️ thank you to all the community members who joined us over the past few months. It’s been an amazing experience sharing this time with you! Stay tuned to our Meetup page, and we’ll see you all next semester : )

• 🏃‍♀️ Speed run some basic knowledge

  • Play and visualise LLMs with LLM Visualization created by Brendan Bycroft.
  • Enjoy transformer videos made by 3Blue1Brown:
    • But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning
    • Attention in transformers, visually explained | Chapter 6, Deep Learning
  • Read these awesome articles from real human intelligence 📜
    • LLM Basics: Embedding Spaces - Transformer Token Vectors Are Not Points in Space from Nicky Pochinkov.
    • A Gentle Introduction to Positional Encoding in Transformer Models, Part 1 from Jason Brownlee.

• 🛠️ Build one from scratch

  • Follow one of tutorial videos from Andrej Karpathy (former OpenAI research scientist):
    • Let's reproduce GPT-2 (124M)
    • Let's build the GPT Tokenizer
    • Let's build GPT: from scratch, in code, spelled out.

• 📜 Read some simple yet functional repos

  • minGPT: A small, clean, interpretable and educational GPT re-implementated in PyTorch.
  • nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs. A rewrite of minGPT.
  • build-nanogpt: Walk through step-by-step and clean GitHub commits to slowly build a nanoGPT.
  • nano-llama31: A minimal, dependency-free implementation of the Llama 3.1 architecture.

• ⚔️ Compare performance of the latest LLMs
  • LLM Leaderboard from vellum
  • Independent analysis of AI models and API providers from Artifical Analysis
  • The LLM Index from Sapling
  • open-llm-leaderboard from Open LLM Leaderboard
  • The Big Benchmarks Collection from Open LLM Leaderboard

• 🎮 Good visualisation is all you need
  • WizMap from Polo Club of Data Science @ Georgia Tech for visualising large-scale token embeddings.
  • Dodrio from Polo Club of Data Science @ Georgia Tech for attention head summarization and semantic and syntactic knowledge contexts from transformer models.

• 📦 Interesting topics and other stuffs
  • ChatGPT: 30 Year History | How AI Learned to Talk from Art of the Problem on YouTube.
  • The moment we stopped understanding AI [AlexNet] from Welch Lab on YouTube.
  • CNN Explainer from Polo Club of Data Science @ Georgia Tech for helping non-experts learn about Convolutional Neural Networks (CNNs).
  • NeuroCartography and Summit from Polo Club of Data Science @ Georgia Tech for visualising image embeddings from ImageNet.

• Data Source Contributor 🕵️‍♀️
  • Identify and provide access to Australia-related data sources.
  • Collaborate with other contributors to ensure data quality and relevance.
• Data Collecting, Crawling and Scraping 👩‍🌾
  • Develop scripts and tools to collect data from various sources.
  • (Optional) Have experience with web scraping tools (e.g., BeautifulSoup, Scrapy).
• Data Cleaning 👩‍⚕️
  • Clean and preprocess datasets to ensure they are ready for analysis and modeling.
  • (Optional) Have experience with data manipulation libraries (e.g., Pandas, NumPy).
• Model Building, Training and Tuning 👩‍💻
  • Develop and train LLMs to solve with our datasets.
  • Have experience with machine learning frameworks (e.g., TensorFlow, PyTorch).
• GitHub Organising 👩‍🔧
  • Manage the GitHub repository by organizing files, documentation, and issues.
  • (Optional) Have proficiency in using Git and GitHub.
• Hugging Face Organising 👩‍🏭
  • Manage and organize model versions and datasets.
  • Ensure proper documentation and metadata for each model and dataset.
• Social Media Organising 👩‍💼
  • Promote the project and its updates on social media platforms (e.g., Discord, Meetup).
  • Engage with the community to increase project visibility and collaboration.

Can't wait to join us? Send a message to our lovely team members:

• Mattew: Matthew.Altenburg@anu.edu.au
• Mohan: MohanBalaji.Paranthaman@anu.edu.au
• Roshan: RoshanRam.Deenadayalan@anu.edu.au