Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerized setup for sonar #19

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Use the latest LTS (Long-Term Support) version of Ubuntu
FROM ubuntu:latest

# Set working directory to /sonar
WORKDIR /sonar

#Install base utilities and Python with required dependencies
RUN apt-get update && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install -y \
build-essential \
wget \
python3.11 \
python3-pip \
python3-venv \
openmpi-bin \
libopenmpi-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Copy requirements.txt to the working directory
COPY requirements.txt .

# Create a virtual environment and install dependencies
RUN python3 -m venv env && \
. env/bin/activate && \
env/bin/pip install -r requirements.txt

# To activate the virtual environment by default, use ENTRYPOINT
ENTRYPOINT ["/bin/bash", "-c", "source /sonar/env/bin/activate && exec \"$@\"", "--"]
1 change: 1 addition & 0 deletions docker_run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
docker run -it --rm -v $(pwd)/src:/sonar/src sonar_image /bin/bash -c "cd /sonar/src && mpirun --allow-run-as-root -np 4 -host localhost:11 ../env/bin/python main.py"
25 changes: 12 additions & 13 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,18 @@ mpi4py==3.1.6
mpmath==1.3.0
networkx==3.3
numpy==1.26.4
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.20.5
nvidia-nvjitlink-cu12==12.5.40
nvidia-nvtx-cu12==12.1.105
nvidia-cublas-cu12
nvidia-cuda-cupti-cu12>=12.1.105
nvidia-cuda-nvrtc-cu12>=12.1.105
nvidia-cuda-runtime-cu12>=12.1.105
nvidia-cudnn-cu12>=8.9.2.26
nvidia-cufft-cu12>=11.0.2.54
nvidia-curand-cu12>=10.3.2.106
nvidia-cusolver-cu12>=11.4.5.107
nvidia-cusparse-cu12>=12.1.0.106
nvidia-nccl-cu12>=2.20.5
nvidia-nvjitlink-cu12>=12.5.40
nvidia-nvtx-cu12>=12.1.105
ogb==1.3.6
outdated==0.2.2
packaging==24.0
Expand All @@ -57,7 +57,6 @@ tifffile==2024.5.22
torch==2.3.0
torchvision==0.18.0
tqdm==4.66.4
triton==2.3.0
typing_extensions==4.12.0
tzdata==2024.1
urllib3==2.2.1
Expand Down
42 changes: 42 additions & 0 deletions src/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,45 @@
## Setting Up the Project with Docker

### Prerequisites

Ensure you have Docker installed on your machine. You can download it from [Docker's official website](https://www.docker.com/get-started).

### Building the Docker Image

1. Clone the repository:
```sh
git clone https://github.com/aidecentralized/sonar
cd sonar
```

2. Build the Docker image:
```sh
docker build -t sonar_image:latest .
```

### Running the Container

We provide a `docker_run.sh` script to simplify running the Docker container.

1. Ensure the script has execution permissions:
```sh
chmod +x docker_run.sh
```

2. Run the script:
- Using './':
```sh
./docker_run.sh
```
- Using 'bash':
```sh
bash docker_run.sh
```

The `docker_run.sh` script will handle the necessary Docker commands to start the container with the appropriate settings.

Note: If you have used a different Image name while building Docker Image, Please update the name accordingly in `docker_run.sh` script

### Running the code
Let's say you want to run the model training of 3 nodes on a machine. That means there will be 4 nodes in total because there is 1 more node in addition to the clients --- server.
The whole point of this project is to eventually transition to a distributed system where each node can be a separate machine and a server is simply another node. But for now, this is how things are done.
Expand Down
Loading