Trust transfer-fit simulations when rollouts use verified events, legal possessions, VAEP-style value, lineup context, many samples, and a post-transfer holdout. Otherwise the model can recommend a player from sequences that could not happen.

ScoutGPT turns transfer fit into a counterfactual possession problem. Hong et al.’s March 16th arXiv paper frames a transfer as a change in event sequences under a hypothetical lineup, with Monte Carlo sampling used to estimate unobserved scenarios.

The full ScoutGPT paper reports 2,283 K League matches from 2021 to 2025, split into 222,940 episodes and 1,490 players. Each episode starts with a 56-token context block for lineups, period and score, then uses 10-token event tuples for team, position, player, action, locations, time delta and outcome. ¹

The six checks

Before a simulated fit reaches a shortlist meeting, the analyst should be able to point to six pieces of evidence.

Football validity first

The VERSA paper found logical inconsistencies in 18.81 percent of Bepro K League 1 events from 2024. Its examples include a block recorded before the shot it stopped and a pass followed by a shot with the missing pass reception inserted later. ²

ScoutGPT uses VERSA preprocessing before training, then applies state-dependent logit masking during generation. Team tokens are restricted to the teams in the episode, action tokens follow VERSA transition rules, pitch coordinates stay inside the pitch, and semantic stopping rules end an episode at a goal, a new episode or an episode-ending action.

Sampler restrictions matter because a plausible action type can still create an illegal possession. A generated block, interception or reception has to respect which team controls the ball, which player acted last, where the event happened, and whether the possession phase has already ended.

Representation changes credit

SPADL documentation defines each on-ball action with game, period, time, player, team, start and end location, action type, result and body part. Those fields let a model assign who acted, where the ball moved and whether the action succeeded. ³

Atomic-SPADL documentation splits a completed pass into the pass and the receival, then adds an interception or out event when the pass fails. A winger receiving between the lines can gain credit in Atomic-SPADL that a one-action pass representation would place on the passer. ⁴

Recruitment simulations built on different action grammars will assign value differently. The analyst needs the event schema before comparing two simulated fits, because the same move can credit the passer, receiver, defender or restart boundary depending on the representation.

Value and context

VAEP documentation values an action through the change in scoring probability minus the change in conceding probability over the next 10 actions. ScoutGPT adds goal-scored and goal-conceded heads at outcome tokens, so each generated state carries a value signal. ⁵

ScoutGPT’s 15-second goal prediction table shows a narrow split. CatBoost has the stronger goals-scored AUC, while ScoutGPT has the stronger goals-conceded AUC and lower goals-conceded Brier score.

A transfer changes who is available for the next pass as well as which player receives it. ScoutGPT’s context block encodes the 22 players, their roles, the period and the score before generation, then uses spatial-aware entity resolution to select the player matching the generated team, position and event location.

Sampling and holdout

ScoutGPT’s Figure 2 reports decreasing mean absolute VAEP deltas as sample count increases in self-to-self simulation. A single rollout can look fluent by chance, so recruitment use needs the spread across repeated rollouts.

The EventGPT full text uses the same player-conditioned family of ideas with learned player embeddings and repeated generation in transfer settings. The Forecasting Events paper says sampler restrictions are needed to avoid unrealistic values during millions of inferences. ^{6 7}

ScoutGPT’s strongest recruitment result is the 2025 transfer holdout. Table 7 reports a lower average absolute error for simulated post-transfer episode VAEP than for a naive previous-season projection across the top 40 transferred players by minutes.

Hoik Jang is the useful warning inside the positive average. The simulated estimate is slightly worse than the naive projection for that displayed case, so the result supports scenario evidence across a holdout group rather than automatic trust in every individual projection.

On-ball only and provider-dependent

The PLOS One event-detection study warns that provider definitions vary, with recovery, crosses, passes and shots all affected by annotation choices. Its independent event and positional data test produced F-scores of 0.71 for passes and 0.65 for shots. ⁸

The Wyscout public dataset paper describes expert operator tagging, automatic checks for missed events, and searches for impossible event-sequence combinations. Professional event data still needs quality control before a model learns from it. ⁹

The evidence stops at K League on-ball event data through 2025 for ScoutGPT’s transfer test. The paper’s conclusion says future work will add tracking data to model off-ball behaviour, which means current rollouts cannot see runs that open the pass, marking that blocks it, or spacing that changes the receiver’s first touch.

The audit line

Clubs should request generated sequences, validity masks, the action schema, value targets, rollout distributions and post-transfer holdout errors before using a simulated fit in recruitment. A demo clip without those records can still be a useful prompt for scouts, video analysts and coaches.

Pause each generated pass, reception, defensive action and stoppage. The analyst must record ball control, VAEP credit, invalid masked options, rollout spread and the post-transfer error line. For ScoutGPT’s top 40 transferred players, that line is 1.25.