BoilerClasses

boilerclasses.v3.demo.mp4

Structure

BoilerClasses is a simple Next.js app with a few Python helper files to format and organize the data. We use a Redis instance to store and rapidly query all our data.

We use Fly.io through Docker to host our app. More steps to run the Docker container through our Dockerfile can be found below.

Setup

You can clone this repository and run a local instance of the app in two ways (with or without Docker):

With Docker

Make sure you have docker installed and the daemon running. More information about installation can be found here. Once you get that up and running, navigate into the cloned repository and run:

docker build . -t boilerclasses

After the image is created, run:

docker run -it -p 3000:3000 boilerclasses

This will expose the container's port 3000 to your machine. Navigate to localhost:3000 to view the app! You can edit whatever files you want locally, but you'll have to rebuild the image every time you want to view your changes. Thus, not ideal for quick changes.

Without Docker

1. Firstly, make sure you have python, node, and redis installed.

2. Then, navigate into the server directory and run:

   sh fetch_snapshot.sh
   

   This downloads the data snapshot pinned in data.lock (the same data production serves) to server/classes_out.json and verifies its hash.

3. Now, you want to spawn a Redis instance at the port 6379. To do this, run the following command:

   redis-server --daemonize yes
   

   The daemonize argument will make it run in the background. Alternatively, if you have docker but don't want to install redis-server, you can run:

   docker run --name boilerclasses-redis -i --rm -p 6379:6379 redis/redis-stack-server:latest redis-stack-server --save
   

   Functionally, both of the above commands are equivalent.

4. Once you have that, you can push all the data from the JSON file generated in step 2 to the Redis instance. To do this, run:

   python3 push.py
   

5. Now, navigate back to the root directory and run:

   npm install
   npm run dev
   

   Now, you can make changes within the Next.js app and have them reflect in real-time at localhost:3000.

   PS: if you look at the Dockerfile, you can see that these exact commands are run!

Data Collection

Data is refreshed by the Data Pipeline GitHub Action (.github/workflows/data.yml), weekly on Mondays or by hand from the Actions tab:

1. scrape.py scrapes the newest 2 semesters from Purdue's catalog and uploads them to our S3 bucket. Older semesters stay as they are on S3.
2. grades.py converts the newest 5 semesters of grade CSVs from BoilerGrades and uploads any that changed.
3. harmonize.py combines everything on S3 into one JSON file and writes src/data/terms.json for the frontend.
4. If the result changed, it's uploaded as snapshots/<sha256>.json, the hash is committed to data.lock, and the site is deployed. Only the snapshots from the last 10 data.lock commits are kept.

To roll back data, revert the data.lock commit. push.py loads classes_out.json into Redis when the container starts.

Manual runs take two optional inputs: terms (e.g. ["Fall 2026"], or [] to skip scraping) and subjects (e.g. CS, a quick test scrape that uploads nothing).

Running the scrape.py script may cause issues, but feel free to tweak line ~42, where the driver is initialized. It is somewhat system-dependent -- that configuration should work on MacOS with a Google Chrome driver and selenium v4.x. If you want more clarification/help, open up an issue!

Future Improvements

We're trying to integrate as many features as possible, and we'll have open issues for the same. If you find a bug or have any feedback, let us through a PR or our feedback form. All contributions are very, very welcome!

Acknowledgements

Inspired by classes.wtf and Purdue's slow course catalogs. We'd like to also thank our friends over at Boilerexams, Purdue.io, and BoilerGrades.