05_summary_next_steps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e586d9f3-86d9-4d4c-bdf4-4f65bf0e9236",
   "metadata": {},
   "source": [
    "# MLOps for RSEs - 5. Summary and Next Steps\n",
    "![Met Office Logo](https://www.metoffice.gov.uk/webfiles/1661941781161/images/icons/social-icons/default_card_315.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dc18d8e-8a74-42ef-bfdd-f15147ed746c",
   "metadata": {},
   "source": [
    "## What have we not covered \n",
    "We've not gone into many aspects of RSE activities in a machine learning lifecycle\n",
    "\n",
    "* Specifics of deploying on public/private cloud platforms \n",
    "* Integrating into a CI/CD pipeline\n",
    "* Specifics of testing\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c86876fe-8ae4-463c-a27f-c8c69d3144d6",
   "metadata": {},
   "source": [
    "## Summary \n",
    "\n",
    "![XKCD 844 good code](https://imgs.xkcd.com/comics/good_code.png)\n",
    "\n",
    "I'm afraid I don't have a general answer for how to write good code, but hopefully this tutorial has given you some good pointers to get started with creating good machine learning workflows for research projects.\n",
    "\n",
    "In this tutorial, we have gone through the key steps of a workflow for a typical machine learning project, which can be divided roughly into the following three steps:\n",
    "\n",
    "* data preparation and exploration\n",
    "* model training\n",
    "* model evaluation and explainability\n",
    "\n",
    "Perhaps the most important point about applying good development practices to a machine learning project is all the same principle and practices that research software engineers apply to other sorts of projects apply equally to ML projects. There are be some additional tools to help apply these principle to the additional assets (e.g. trained model weights) and specific workflows (e.g. experiment tracking), but generally all same principles and practices apply, such as:\n",
    "\n",
    "* *version control* - Use version control for all assets, including code, data and models.\n",
    "* *FAIR principles (Findable, Accessible, Interoperable, Reusable) * - Ensure you store models as well as code and data according to FAIR principles [FAIR principles](https://www.go-fair.org/fair-principles/)\n",
    "* *ARCO (Analysis-ready cloud optimised) - Ensure your data is is available in a form that is ready for use in machine learning pipelines with minimal transformation required. Also ensure it is stored in a format that is optimised for the platform on which it will be used and supports parallel access usage patterns.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85de2bec-6b19-4d4f-a8e7-b4b91e98aa77",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Next Steps - Projects\n",
    "\n",
    "The key question (hopefully) after such a tutorial is \"how can I make use of what I have learned in my projects now and in the future?\". Here are some initial steps in applying MLOps principles and practices that you can take immediately:\n",
    "\n",
    "* Encourage researcher to use Experiment Tracking software (e.g. ML Flow)\n",
    "* Review how your assets are organised and shared. Can you create a catalog of assets to make it easier for researchers to use them?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b52e826-9c60-487a-888c-5c4c9afb16e6",
   "metadata": {},
   "source": [
    "## Next Steps - Personal Development \n",
    "There are also next steps for you as a Research Software Engineer around developing yourself:\n",
    "* Take the deeplearning.ai [MLOps course on coursera](https://www.coursera.org/specializations/machine-learning-engineering-for-production-mlops)\n",
    "* Go through tutorials for key tools such as\n",
    "  * ML Flow\n",
    "  * Ray\n",
    "  * Tensorboard\n",
    "  * Intake\n",
    "  * Holoviz\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}