LogoLogo
TournamentsHedge FundDiscordGitHubForumDocs
  • Numerai Tournament
    • Overview
    • Data
    • Models
    • Submissions
      • Model Uploads
      • Compute Heavy
      • NGROK Example
      • CRON Example
    • Scoring
      • Definitions
      • Correlation (CORR)
      • Meta Model Contribution (MMC)
      • Feature Neutral Correlation (FNC)
      • Grandmasters & Seasons
    • Staking
    • Bounties
  • Numerai Signals
    • Overview
    • Data
    • Models
    • Submissions
    • Scoring
      • Definitions
    • Staking
    • Signals + QuantConnect
  • Numerai Crypto
    • Overview
    • Data
    • Submissions
    • Scoring
      • Definitions
    • Staking
  • Community
    • Discord
    • Forum
    • Twitter
    • Youtube
    • Apps & Content
      • Office Hours with Arbitrage
        • Office Hours Recaps: Season 1
          • OHwA S01E01
          • OHwA S01E02
          • OHwA S01E03
          • OHwA S01E04
          • OHwA S01E05
          • OHwA S01E06
          • OHwA S01E07
          • OHwA S01E08
          • OHwA S01E09
          • OHwA S01E10
          • OHwA S01E11
          • OHwA S01E12
        • Office Hours Recaps: Season 2
          • OHwA S02E01
          • OHwA S02E02
          • OHwA S02E03
          • OHwA S02E04
          • OHwA S02E05
          • OHwA S02E06
          • OHwA S02E07
          • OHwA S02E08
          • OHwA S02E09
          • OHwA S02E10
          • OHwA S02E11
        • Office Hours Recaps: Season 3
          • OHwA S03E01
          • OHwA S03E02
          • OHwA S03E03
          • OHwA S03E04
        • Office Hours Season 4
        • FAQ in Office Hours
        • Cited resources
      • Structure of Numerai by Wigglemuse
  • NMR
    • Coinbase
    • Uniswap
    • Etherscan
    • Github
  • Connect
    • Index
Powered by GitBook
On this page
  1. Community
  2. Apps & Content
  3. Office Hours with Arbitrage
  4. Office Hours Recaps: Season 2

OHwA S02E09

From August 6, 2020 / Wigglemuse interview

PreviousOHwA S02E08NextOHwA S02E10

Last updated 3 years ago

For this, the ninth episode of season two, Arbitrage interviewed longtime tournament participant and all-start .

The full interview with Wigglemuse will be published on YouTube.

Questions from Slido

Thanks to the overwhelming crowd response, NJ agreed to come on Office Hours as a guest on August 27th, "assuming nothing groundbreaking with the tournament happens between now and then," she said.

Arbitrage: Does that mean there's something coming??

NJ: No.

Arbitrage: I tried 🤷♂

Assuming a reasonably fit model, how much extra performance can one expect from training on validation? Is it even worth it?

Ultimately, whether or not training a model on the additional data will improve that model's performance is dependent on the current regime.

How have the new accounts been performing? Does it look like more experimentation is happening or is it just the same, more or less?

When asked about the changes to the meta-model the team has mentioned, Michael Oliver explained that they implemented a change, but it wasn't nearly as drastic as they originally anticipated.

What's a good value for feature exposure?

Michael Oliver said that if your feature exposure is too high, you'll have more variance, and if it's too low your model will have too low of a mean. He added that if you're interested in feature exposure, look at the distribution of your scores and find your maximum feature exposure. "Each feature exposure is a risk," Michael Oliver said, "and if you still have some very high risks, that might get covered up by other ones you're low on."

Will a shift come soon?

Assuming a shift refers to a burn period, Arbitrage said that he believes that at the time of this episode, it was already beginning. This was anecdotally based on his performance trending downward over the previous 20 weeks and suddenly changing, suggesting that a regime change took place.

Are the scores we see on the tournament website a day older than we realized? Are Wednesday's scores from Monday's market close?

Numerai software engineer Jason confirmed that Wednesday's finish is Monday's close, and Thursday is Monday close through Tuesday close.

When ?

Arbitrage used his as an example, which only saw the first 120 eras and performs relatively well on the live data. The question is asking if Arbitrage were to take that model with the same parameters, settings, and configurations but add the remaining eras, will it perform better?

Arbitrage explained that he and do something very similar. They look at eras 1 - 132 and noticed that they have a small amount of additional variance compared to similar models trained on eras 1 - 120 (although Themicon noted that during the ongoing round at the time of recording, his model trained on validation was outperforming those that were not).

Since , noted that generally tournament health has improved and there's slightly more variance among models.

Michael Oliver: There are some weird models that people submit. I'm not even sure that they're training these models; they're just transforms of the data. I clearly don't think that's a good idea... As is always telling people, just because something is unique, doesn't mean it's good. We want different and good. There are infinite ways to be different and bad, and so we don't want those models. Stake weighting still seems to be working quite well, there's been a good feedback loop.

Arbitrage the person shared that has a feature exposure of 0.923 (Author's note: for comparison, my model is rank 623 at the time of writing and has a feature exposure of 0.0754). Michael Oliver shared some of his feature exposures: = 0.0855, = 0.0823, = 0.076.

If you’re passionate about finance, machine learning, or data science and you’re not competing in_, what are you waiting for?

Don’t miss the next Office Hours with Arbitrage : follow_ or join the discussion on _for the next time and date.

Thank you to_ , , and for contributing to answers during this Office Hours, to Wigglemuse for being interviewed and his presence in , and to for hosting.

NJ
Arbitrage 2
Themicon
accounts were expanded to support up to ten models
Michael Oliver
Mike P
Arbitrage the model
MDO
NMRO
NIAM
the most challenging data science tournament in the world
Numerai on Twitter
Rocket.Chat
Michael Oliver
Themicon
Wigglemuse
Rocket.Chat
Arbitrage
Rocket.Chat
Wigglemuse
Wigglemuse gets the center square, the Square of Honor