iproxy

Built your own IPV4 geolocation database

Built with ❤ in Rust

Table of Contents generated with mtoc

• Badges
• Introduction
• How works
• Example
• Using the API
  • Config.toml
  • Launch the docker-compose
  • Run your first query
  • Api alive?
  • Stop the stack
• Local development
  • Stop local mongodb
• Visualize data
• Import from local mongodb to mongodb atlas
  • Install mongodb tools
  • Dump local database
  • Import local database to mongodb atlas
• TO DO
• Pending to fix
• Useful Links
• LICENSE

Badges

Introduction

So imagine that you want to start indexing public IPv4 geolocation data. There are paid services, others free but requiring registration, that offer this type of content. By consulting one of these services you will obtain the data you need.

I do not know exactly all the public services that offer this data, in my case I have used the public database of ip-api.com

Without this external service this tool does not work, it would be necessary to refactor and use another API and map the responses.

Other possible solutions (not implemented):

• https://www.maxmind.com/en/home
• https://ipstack.com/
• https://ip.guide

How works

You will make a request to your API endpoint, for example, curl http://127.0.0.1/api/v1/1.1.1.1, and the API will first check if the data exists in the database. If it does not exist, it will retrieve the information from the external website mentioned above. The next time you query the same IP, the data will be retrieved from MongoDB, avoiding the external query.

Example

API logs can be retrieved executing docker logs -f iproxy once the API is running. Plase visit the next step.

Using the API

The setup is configured to work with docker-compose locally.

Config.toml

With this file located in the root of this repository, you will be able to change some parameters.

Launch the docker-compose

⚠️ WARNING
Before start the docker-compose, change the directory where you want to save the mongodb data

Example, from compose.yml:

  mongodb:
    ....other config
    volumes:
      - /mnt/ssd/iproxy:/data/db # this line!!

In mi case, I'm mapping all the data using an external SSD mounted in /mnt/ssd

Now, launch all the stack:

docker network create iproxy
make compose-up-build

This will starts the iproxy container and mongodb.

Run your first query

curl http://127.0.0.1:8000/api/v1/1.1.1.1

http://ip:port/api/v1/ip-to-query

Api alive?

curl http://127.0.0.1:8000/api/v1/health

Stop the stack

make compose-down

Local development

cargo install cargo-watch systemfd
docker network create iproxy
make local-development

Stop local mongodb

make local-development-down

Visualize data

Using mongodb compass you can visualize your data from the collection ips

Import from local mongodb to mongodb atlas

Import data from localhost to mongodb atlas

Install mongodb tools

Visit the official webpage https://www.mongodb.com/docs/database-tools/installation/installation/.

Dump local database

mongodump --uri="mongodb://admin:admin@localhost:27017/?maxPoolSize=20&w=majority"

This command will create a new dump/ directory with the backup

Import local database to mongodb atlas

The uri of this command is the remote uri (mongodb atlas)

mongorestore --uri="mongodb+srv://USERNAME:PASSWORD@XXXXX.XXXX.mongodb.net/?retryWrites=true&w=majority" --db="ipfinder" --collection="ips" dump/iproxy/ips.bson

TO DO

• Testing and error handling with custom errors (MyErrors)
• Generate possible public ipv4 https://www.criminalip.io/ip-ranges
• Prometheus metrics
• JWT token

Pending to fix

• When the ip is private, the process crashes.

Useful Links

• Github
• API example
• API example mongodb

LICENSE

iproxy is distributed under the terms of the GNU AFFERO GENERAL PUBLIC LICENSE.