Databricks project troubleshooting

Find the message from smoke.py, preview.py or the failed task below. The index points to the page that owns each fix. The three detailed diagnostics at the end come from the project validators. You can reproduce them locally without Spark, credentials or a deployed job. Use the Core release matching your generated project.

Each correction below is a change to an existing generated project, not a complete replacement configuration. Start a project with the walkthrough. See the glossary for the lifecycle and data terms.

Find your message

Search this table for the start of your message. Values in angle brackets vary. The linked section explains the cause and the fix.

Message or symptom	Where to fix it
Pre-split step '<step>' reads target '<target>'. Set SKIP_TARGET_PRE_SPLIT_STEPS=True or SCORING_MODE='custom'.	Below
feature_lookup.lookups[<n>].table_name: Unity Catalog names must be catalog.schema.name; each part a simple identifier (letters, digits, underscore; not starting with a digit).	Below
feature_lookup.lookups[<n>] is not filled in: complete feature_lookup and online_features in config/training.yml, or regenerate with include_online_feature_lookup=false.	Below
Automatic promotion requires quality_threshold.	Below
AliasOutcomeUnknownError, or a pending alias event	Lifecycle recovery
Project and job definitions disagree; regenerate/redeploy the Bundle together. after a Core or template upgrade (workflow contract)	Project upgrade
The same message after changing follow_champion in config/inference.yml	Redeploy the Bundle; see score handoff
Training source has ... rows, exceeding max_rows=..., Training source observed size ... bytes exceeds max_bytes=... or Training source estimated size ... bytes exceeds max_bytes=...	Training data: adjust max_rows / max_input_mb in config/training.yml, or set training_sample_rows to sample eligible rows before splitting.
Training input has ... rows, exceeding max_rows=..., Training input size ... bytes exceeds max_bytes=... or Competition training data ... exceeds max_rows=... / max_bytes=...	Python API: reduce the input size or adjust max_rows / max_bytes in fit_workflow or evaluate_competition_candidate. The frame training limit covers all splits together. Increase memory limits only when the process has enough memory.
Initial scoring source exceeds the configured max_rows budget. or Source increment exceeds max_rows.	Scoring
<node>.apply on pandas: <reason>, for example project_scoring.apply on pandas: Custom scoring and reused pre-split rules are unsupported. (Spark admission)	Distributed inference
An online endpoint returns Encountered an unexpected error while evaluating the model. ... Error ''	Online features: read the served model logs
A Unity Catalog permission error in a job run	Deployment; test the job's run_as identity
Set both monitoring_catalog and monitoring_schema.	Set both in deployment/variables.yml, or leave both blank with monitoring_enabled: "false". See if no destination is set
Local artifact runtime mismatch for skyulf-core: requires ..., found ... or report reason champion_runtime_unavailable	The model was produced by another runtime. Approve a reviewed replacement with allow_unloadable_champion=true, or run with the producing release. See champion from an older Core release.
bundle init fails with ERROR_monitoring_schema_is_required_with_monitoring_catalog	A monitoring catalog was answered without a schema. Answer both, or leave both blank to start without monitoring. See project setup.
Spark reference is not prepared; run prepare_spark_monitoring_reference first.	Monitoring enrollment
Serving endpoint is not ready or its config update is pending.	Serving

Common errors on each topic page

Topic	Typical symptoms
Project setup	Wrong schema in preview, missing columns or notebooks, monitoring destination
Deployment	REPLACE_* values, authentication, privileges, duplicate jobs
Project upgrade	Graph contract, missing notebooks, saved model runtime
Training data	Budgets, temporal boundaries, null labels, timezones
Custom steps	Factory paths, row count, source package and dependencies
Custom pre-split filters	Target-reading filters, non-Boolean masks
Decision thresholds	Too few classes, unknown manual class, threshold conflicts
Training reports	Missing report tasks, charts or SHAP, truncated parameters
Explanations	Missing SHAP tasks or artifacts, explanation status codes
Model sets	Replacement stays challenger, quality failures, Spark admission
Approve, reject and roll back	Missing quality threshold, changed champion, scoring after promotion
Lifecycle recovery	Uncertain alias changes
Scoring	Champion selection, eligibility, budgets, schema
Monitoring overview	Monitoring destination, nothing monitored, old dashboard results
Monitoring store setup	Store permissions and upgrades
Monitoring enrollment	Enrollment and references
Delayed labels	Label columns, timestamps, duplicate keys, classes
Drift	no_data, degraded, large PSI without drift
Performance	Unknown policy fields, missing baseline, failure streaks
Retraining	Why no new train run appeared
Monitoring dashboard	Issues to investigate, missing links
Serving	Endpoint readiness and privileges
Online features	Missing and stale online features
Serving monitoring	awaiting_logs, failed, blank counts or latency
Daily rollout	Retries, stops and recovery of traffic updates
Remove a project	Unexpected deletions, shared dashboard, endpoints left running

A pre-split step reads the target

Example message, where the step name and target depend on your recipe:

Pre-split step 'dropmissingrows' reads target 'churn'. Set SKIP_TARGET_PRE_SPLIT_STEPS=True or SCORING_MODE='custom'.

Cause. The selected config/pre_split.yml recipe filters or edits churn, the actual target. That can be appropriate when choosing labeled training rows. The default scoring mode also reuses this recipe, but prediction requests do not contain the actual target. Project loading resolves the saved scoring policy, so this can fail during the training preview, before fitting or scoring.

Fix. Decide the scoring policy in config/scoring.yml before preview and training. To reuse the feature checks while omitting target-dependent rules, set:

version: 1
mode: pre_split
skip_target_pre_split_steps: true
eligibility: []
outputs: []
combined: []

A filter reading both target and feature columns is skipped as a whole. Split it into separate target and feature checks if the feature check must still apply to scoring. Fixed edits can instead be projected onto the selected input columns.

Alternatively, select mode: custom with skip_target_pre_split_steps: false and define the required eligibility and output rules in the same file. Custom mode does not reuse pre-split rules; the generated custom lists start empty. Do not select it expecting the commented examples to become active. The training pre-split recipe still runs in either case. These choices are saved with the newly fitted model; editing the file does not change an already registered version's scoring policy.

Check. From the project root, run:

python src/tools/smoke.py
python src/tools/preview.py --action train
python src/tools/preview.py --action score

Confirm the selected recipe and scoring mode, then train a new model version. Read pre-split admission and scoring and scoring eligibility.

A feature lookup table name is incomplete

Example message for the second lookup:

feature_lookup.lookups[1].table_name: Unity Catalog names must be catalog.schema.name; each part a simple identifier (letters, digits, underscore; not starting with a digit).

Cause. feature_lookup.lookups[N].table_name is blank, contains an unresolved placeholder, has fewer than three parts or has invalid identifier syntax. The index is zero-based: [0] is the first list entry and [1] is the second. A completely blank generated entry (no table_name, lookup_key or feature_names) reports feature_lookup.lookups[N] is not filled in: ... instead: fill the entry, or regenerate the project with include_online_feature_lookup=false. These checks validate names locally; passing them does not prove the table exists or that the job can read it.

Fix. In config/training.yml, find the effective feature_lookup under defaults or the model override and repair the indexed entry's table_name. For example, replace an empty second table name with this field value:

table_name: "{catalog}.{input_schema}.balances"

Here catalog and input_schema come from the selected project bindings. A concrete workspace.features.balances also works. Use your own existing feature table name and simple identifiers. Preserve the entry's lookup_key, feature_names and any timestamp lookup settings; ensure they match the table's keys and the model's declared inputs. A model override replaces the complete lookup mapping, so check the effective model entry rather than repairing only a shadowed default. The syntax correction does not create or merge a feature table.

Check. Run the three local checks above with the intended catalog/schema bindings. The expanded table name should resolve without this message. Then verify table existence, feature metadata and permissions in the chosen workspace before training. See feature tables and lookup choices and the training reference.

Automatic promotion has no quality threshold

Automatic promotion requires quality_threshold.

Cause. The effective model configuration has promotion_policy: automatic and a missing or null quality_threshold. A relative improvement requirement or additional quality_gates does not replace the primary absolute threshold. The validator stops before remote training and registry work.

Fix. In config/training.yml, set a numeric threshold under defaults or the affected model entry, using the units and direction of its metric. For an existing regression model, this is a fragment illustrating an RMSE upper bound:

defaults:
  promotion_policy: automatic
  metric: heldout_rmse
  quality_threshold: 5.0

The value 5.0 is an example, not a recommended business limit. Choose your acceptance criterion: lower RMSE is better, while classification accuracy/F1 needs a minimum acceptable score. Check model overrides for a null that masks the default. Keep any additional gates and min_improvement requirements appropriate to the same target and task. Choosing manual_approval allows a pending candidate without this automatic-policy check; first-champion activation still needs an explicit quality threshold and passing saved evidence.

Check. Run smoke.py and preview.py --action train again. Passing validation only confirms a complete policy. After training, inspect the saved held-out metric and every gate in training_report before accepting the candidate. Read the quality fields and registry lifecycle.

If your message is different

Keep the full traceback, task key, model/branch name, concrete model version and the effective configuration for the failing action. Inspect the first failing task; excluded conditional tasks are often expected. Retry only after checking whether training, alias changes or prediction publication already committed. Approve, reject and roll back explains saved results and operator actions; scoring recovery covers the recognized CDF-expiry path. This page does not enumerate every validator or imply that a local preview verifies cloud execution.