update dataset via api from python - CSRF token

[mestachs]

I tried something like this get the bearer token

    def _login(self, username, password):
        url = f"{self.base_url}/api/v1/security/login"
        payload = {
            "provider": "db",
            "refresh": True,
            "username": username,
            "password": password,
        }
        r = self.s.post(url, json=payload)
        r.raise_for_status()
        access = r.json()["access_token"]

        # Use Authorization header with JWT
        self.s.headers.update({
            "Authorization": f"Bearer {access}",
            "Content-Type": "application/json"
        })

then post the payload

url = f"{self.base_url}/api/v1/dataset/{dataset_id}"
resp = self.s.put(url, json=dataset)

but apparently the api complains about 400 Bad Request: The CSRF token is missing.

{"errors": [{"message": "400 Bad Request: The CSRF token is missing.", "error_type": "GENERIC_BACKEND_ERROR", "level": "error", "extra": {"issue_codes": [{"code": 1011, "message": "Issue 1011 - Superset encountered an unexpected error."}]}}]}

is there a way to get one ? (why is it asking for one since I use the Bearer JWT)

[dosubot[bot]]

Hi @mestachs! I'm Dosu and I’m helping the superset team.

Superset enforces CSRF protection on modifying API endpoints (like PUT to /api/v1/dataset/{id}) even when using Bearer JWT authentication, due to the @Protect() decorator on those routes. This means you must include a CSRF token in your requests, not just the Authorization header with your JWT token cite.

To update a dataset via the API from Python, use requests.Session() to manage cookies, and follow this workflow:

1. Log in to get your access token.
2. Add the Authorization header to your session.
3. Fetch the CSRF token from /api/v1/security/csrf_token/ (with the Authorization header set).
4. Add the CSRF token to your session headers as X-CSRFToken.
5. Make your PUT request with the same session.

Example:

import requests

session = requests.Session()

# 1. Login
login_resp = session.post(
    "https://your-superset/api/v1/security/login",
    json={"username": "your_user", "password": "your_pass", "provider": "db"}
)
access_token = login_resp.json()["access_token"]
session.headers.update({"Authorization": f"Bearer {access_token}"})

# 2. Get CSRF token
csrf_resp = session.get("https://your-superset/api/v1/security/csrf_token/")
csrf_token = csrf_resp.json()["result"]
session.headers.update({"X-CSRFToken": csrf_token})

# 3. Update dataset
resp = session.put(
    f"https://your-superset/api/v1/dataset/{dataset_id}",
    json=dataset_payload
)
print(resp.json())

Do not manually set the Cookie header; let requests.Session() handle it cite.

If you get a 422 error from the CSRF endpoint about "Subject must be a string", make sure the 'sub' claim in your JWT is a string, not an integer cite.

There is no supported way to disable CSRF for API endpoints using JWT Bearer tokens in Superset.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[dosubot[bot]]

The validation errors are because the payload from GET includes fields that aren't allowed in the PUT request. For example, the 'owners' field must be a list of integer user IDs, not objects or strings, and columns must not include fields like 'type_generic', 'changed_on', or 'created_on'—these are read-only or computed fields and must be removed reference.

Here's how to fix your payload:

• For 'owners', use a list of user IDs, e.g. "owners": [12, 34].
• For 'columns', only include allowed fields like column_name, description, expression, type, advanced_data_type, verbose_name, uuid, etc. Remove any unknown fields.

Example minimal valid update payload:

{
  "description": "Updated description",
  "owners": [12, 34],
  "columns": [
    {
      "id": 1,
      "column_name": "my_col",
      "type": "VARCHAR",
      "description": "My column description",
      "expression": null,
      "verbose_name": "My Col",
      "uuid": "c0a80101-0000-0000-0000-000000000001"
    }
    // ... more columns
  ]
}

Before sending the PUT request, strip out any fields from columns that aren't in the allowed list: id, column_name, type, advanced_data_type, verbose_name, description, expression, extra, filterable, groupby, is_active, is_dttm, python_date_format, uuid.

If you mutate the GET payload, make sure to clean up these fields or build the update payload from scratch using only the fields above reference.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[mestachs]

is there a python snippet in the code base to remove them automatically ?

[dosubot[bot]]

There isn't a built-in Python snippet or utility in the Superset codebase that automatically strips disallowed fields from dataset payloads before an API update—validation is handled strictly by the Marshmallow schemas, and any extra fields will cause errors (schemas.py).

To help, here's a Python snippet you can use to clean your payload before sending it:

# Allowed fields for dataset and columns (from DatasetPutSchema and DatasetColumnsPutSchema)
DATASET_FIELDS = {
    "table_name", "database_id", "sql", "filter_select_enabled", "fetch_values_predicate",
    "catalog", "schema", "description", "main_dttm_col", "normalize_columns",
    "always_filter_main_dttm", "offset", "default_endpoint", "cache_timeout",
    "is_sqllab_view", "template_params", "owners", "columns", "metrics", "folders",
    "extra", "is_managed_externally", "external_url"
}
COLUMN_FIELDS = {
    "id", "column_name", "type", "advanced_data_type", "verbose_name", "description",
    "expression", "extra", "filterable", "groupby", "is_active", "is_dttm",
    "python_date_format", "uuid"
}

def clean_column(col):
    return {k: v for k, v in col.items() if k in COLUMN_FIELDS}

def clean_dataset_payload(payload):
    cleaned = {k: v for k, v in payload.items() if k in DATASET_FIELDS}
    if "columns" in cleaned:
        cleaned["columns"] = [clean_column(col) for col in cleaned["columns"]]
    if "owners" in cleaned:
        # Ensure owners is a list of integers
        cleaned["owners"] = [int(o) for o in cleaned["owners"] if isinstance(o, int) or (isinstance(o, str) and o.isdigit())]
    return cleaned

Use clean_dataset_payload(your_payload) before sending your PUT request. This will remove fields like created_on, changed_on, type_generic, and any others not accepted by the API schema, helping you avoid validation errors (api.py). If you need to adapt this for metrics or folders, you can extend the same pattern.

Let me know if you want a version that handles nested metrics or other fields!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[mestachs]

metrics also should be cleaned

[dosubot[bot]]

Here's an updated cleaning snippet for metrics. The allowed fields for metrics are: id, expression, description, extra, metric_name, metric_type, d3format, currency, verbose_name, warning_text, and uuid. For the nested currency field, only symbol and symbolPosition are allowed. These are defined in the Superset codebase's DatasetMetricsPutSchema and DatasetMetricCurrencyPutSchema (source).

# Allowed fields for metrics and nested currency
METRIC_FIELDS = {
    "id", "expression", "description", "extra", "metric_name", "metric_type",
    "d3format", "currency", "verbose_name", "warning_text", "uuid"
}
CURRENCY_FIELDS = {"symbol", "symbolPosition"}

def clean_currency(currency):
    if not isinstance(currency, dict):
        return None
    return {k: v for k, v in currency.items() if k in CURRENCY_FIELDS}

def clean_metric(metric):
    cleaned = {k: v for k, v in metric.items() if k in METRIC_FIELDS}
    if "currency" in cleaned:
        cleaned["currency"] = clean_currency(cleaned["currency"])
    return cleaned

def clean_dataset_payload(payload):
    # ... your existing DATASET_FIELDS, COLUMN_FIELDS, clean_column ...
    cleaned = {k: v for k, v in payload.items() if k in DATASET_FIELDS}
    if "columns" in cleaned:
        cleaned["columns"] = [clean_column(col) for col in cleaned["columns"]]
    if "metrics" in cleaned:
        cleaned["metrics"] = [clean_metric(m) for m in cleaned["metrics"]]
    if "owners" in cleaned:
        cleaned["owners"] = [int(o) for o in cleaned["owners"] if isinstance(o, int) or (isinstance(o, str) and o.isdigit())]
    return cleaned

This will ensure your metrics field is API-compliant and avoids marshmallow validation errors.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}