Last updated
Last updated
Model uploads are a simple and free way to automate your daily submissions. You upload a .pkl
file with your trained model and we handle generating and submitting the predictions for you. You don't need to worry about reliability, infrastructure, scheduling, etc. you just focus on data science.
See for a full tutorial or for a barebones example.
Install requirements from the execution environment and train a model. These are strict package version requirements that will help ensure your model always runs correctly. You may try to install later minor versions / patches of the packages listed as long as it is backwards-compatible with this environment.
Anything not listed is not available, but we do take requests over on .
Wrap your model in a function that takes live features and outputs live predictions:
This is the only way for Numerai to properly run your model for you. If your pickle doesn't have a predict function with this signature, it will fail.
In the example above, our predict
function references model
and feature_cols
defined in the global scope. Cloudpickle is smart enough to correctly serialize both model
and feature_cols
by value so that it is also available when this function is run by Numerai.
Select your pickle file, set the Python version you used, then click Upload
Numerai will execute your model to generate a live submission for the current round:
If this succeeds, a block will appear in the "submissions" column grid, you'll be able to review the execution logs for your model, and Numerai will generate diagnostics over the validation dataset for your model:
Once diagnostics complete, you'll also be able to view the diagnostics for this model:
Once uploaded, Numerai runs your model every day to generate and submit live predictions for you. If everything is working correctly, you will see the latest submission cycle through these 4 statuses:
Pending: Numerai is provisioning the cloud resources to run your model
Running: Numerai is now running your model
Validating: Numerai has ran your model and is now validating your submission
Success: Numerai has accepted your submission
If there was a problem, you will see 2 possible statuses:
Error: Numerai has encountered an unexpected error running your model and the team will look into it.
Failed: Your model failed to run. Please check the logs and re-upload a working model. Examples of model failures include
Python or dependency version mismatch
Invalid submission
Out of memory
Timeout
If, at any point, you would like to disable your uploaded model:
Click the Model Upload button:
In the "Settings" tab, select "Disable"
The default version is Python 3.10, but Python 3.9 is supported and can be selected when uploading a model.
No, we do not give your model internet access.
By default, we will provision each model a machine with 1 CPU with 4GB of ram and allow runtime of up to 10 minutes (does not include time spent queueing).
LightGBM Benchmarks:
20K trees using the small feature set runs in under 1 minute
90K trees using the full feature set runs in under 6 minutes
No, once you upload your model, you will no longer be able to upload submissions via the API.
No, to avoid race conditions after you upload your model, you cannot configure any webhooks on it. You will need to disable any existing compute configuration in order to upload your model.
Numerai reserves the right to disable your model for any reason, including (but not limited to) security, abuse, account inactivity, poor performance, etc.
Numerai will try our best to support your usage of this feature, but ultimately it is still your responsibility to make sure your submission pipeline is set up properly. We do not guarantee that your model will submit 100% of the time.
Numerai is not responsible for the gains or losses you might experience due to the performance of your uploaded model. You are solely responsible for the performance of the model.
Pickle your function with the library and upload the pickle file to Numerai:
This library is used in other computing frameworks like and and allows you to easily run your local Python code remotely in the cloud. The main benefit of using cloudpickle over the standard library is that it serializes your local context along with your code. This makes it very convenient to package up code developed locally in any environment.
Upload your Model on the (the Upload Model button on the right):
Visit the
To see which package versions are used in the Model Uploads environment, see the Numerai Predict requirements.txt:
Yes, Numerai has access to and can unpickle your trained model. If you are not comfortable with this, you should consider using .
Free, zero-setup automation