Overview

Everything you need to know to get started in under 5 minutes!

Introduction

Numerai is a data science competition where you build machine learning models to predict the stock market. You are provided with free, high quality data that you can use to train models and submit predictions daily. Numerai computes the performance of these predictions over the following month and you can stake NMR on your model, to earn (or burn) based on your model's performance.

You can sign up and visit your home page for a full suite of tutorials.

Data

Numerai's free dataset is made of clean and regularized financial data. The dataset is obfuscated so that it can be given out for free and modeled without any financial domain knowledge. This also means that models you build on this data cannot be used outside of the Numerai tournament.

Here is an example of the general structure of our dataest:

Each row in the dataset corresponds to a specific stock at a specific point in time. The point in time is noted by the era - each represents a week. The IDs are unique in each era such that you cannot match stocks across eras - this is necessary for the obfuscation. The features are quantitative attributes known about the stock at the time (e.g P/E ratio, ADV, etc.). The target is a measure of stock market returns 20 days into the future where low means bad performance and high means good performance.

Here is an example of how to get our dataset:

from numerapi import NumerAPI
import pandas as pd

napi = NumerAPI()
# Use int8 to save on storage and memory
napi.download_dataset("v4.3/train_int8.parquet")
training_data = pd.read_parquet("v4.3/train_int8.parquet")

See the Data section for more details.

Modeling

Your objective is to build machine learning models to predict the target given the features. You can use any language or framework that you like.

Here is an example model in Python using LightGBM:

import lightgbm as lgb

features = [f for f in training_data.columns if "feature" in f]

model = lgb.LGBMRegressor(
      n_estimators=2000,
      learning_rate=0.01,
      max_depth=5,
      num_leaves=2 ** 5,
      colsample_bytree=0.1
)
model.fit(
      training_data[features],
      training_data["target"]
)

See the Models section for more examples.

Submissions

Each day (Tuesday through Saturday), new live data is released. This represents the current state of the stock market. You must generate live predictions and submit them to Numerai.

Here is an example of how you generate and upload live predictions in Python:

# Use API keys to authenticate
napi = NumerAPI("[your api public id]", "[your api secret key]")

# Download latest live features
napi.download_dataset(f"v4.3/live_int8.parquet")
live_data = pd.read_parquet(f"v4.3/live_int8.parquet")
features = [f for f in live_data.columns if "feature" in f]
live_features = live_data[features]

# Generate live predictions
live_predictions = model.predict(live_features)

# Format and save submission
submission = pd.Series(
    live_predictions, index=live_features.index
).to_frame("prediction")
submission.to_csv(f"submission.csv")

# Upload submission
napi.upload_predictions(f"submission.csv")

This is what a submission might look like:

Behind the scenes, Numerai combines the predictions of all models into the Stake-Weighted Meta Model, which in turn is fed into the Numerai Hedge Fund for trading.

See the Submissions section for more details and examples.

Scoring

Submissions are scored against two main metrics:

Since the target is a measure of 20 business days of stock market returns, it takes about 1 month for each submission to be fully scored.

See the Scoring section for more details.

Staking

When you are ready and confident in your model's performance, you may stake on it with NMR - Numerai's cryptocurrency. After the 20 days of scoring for each submission, models with positive scores are rewarded with more NMR, while those with negative scores have a portion of their staked NMR burned (destroyed such that no one, not even Numerai, can access it).

Staking serves two important functions:

  1. "Skin in the game" allows Numerai to trust the quality of staked predictions.

  2. Payouts and burns continuously improve the weights of the Meta Model.

See the Staking section for more details.

Support

Find us on Discord for questions, support, and feedback!

Last updated