errno

Reading and writing MediaWiki pages from a script: action=raw and the two-token login

· tested on MediaWiki 1.4x, api.php, bot password, Python requests

Reading: skip the HTML, ask for raw wikitext

Fetching an article URL with a normal client and parsing the HTML is where most scripts start, and it goes wrong in a way that is easy to miss: a plain GET of the page URL can land you on Recent Changes instead of the article — a redirect that a browser resolves invisibly but that leaves your parser looking at completely different content and “working”.

Ask for the source directly:

https://wiki.example.com/index.php?title=Release:MyProject&action=raw

That returns clean wikitext, no skin, no navigation, no redirect games. For anything programmatic — diffing a deployed-versions table, checking whether a section exists — this is the endpoint you want.

Writing: login token, then CSRF token, then edit

Editing needs two tokens, fetched in this order, on the same session (cookies matter):

import requests

API = "https://wiki.example.com/api.php"
s = requests.Session()

# 1. login token
lt = s.get(API, params={"action": "query", "meta": "tokens",
                        "type": "login", "format": "json"}).json()
login_token = lt["query"]["tokens"]["logintoken"]

# 2. log in with a BOT password (Special:BotPasswords), not your account password
r = s.post(API, data={"action": "login", "lgname": USER, "lgpassword": PASSWORD,
                      "lgtoken": login_token, "format": "json"}).json()
assert r["login"]["result"] == "Success", r

# 3. CSRF token — only valid after a successful login
ct = s.get(API, params={"action": "query", "meta": "tokens", "format": "json"}).json()
csrf = ct["query"]["tokens"]["csrftoken"]

# 4. edit
e = s.post(API, data={"action": "edit", "title": PAGE, "text": new_wikitext,
                      "summary": "automated update", "bot": 1,
                      "token": csrf, "format": "json"}).json()
assert "edit" in e and e["edit"]["result"] == "Success", e

Things that bite here:

The trap that actually cost me the time: the credential file

The credentials lived in a per-service file in my credential store, and I wrote the obvious loader: first line is the user, second the password. It failed with a login error, and I spent the next while debugging the token dance — which was correct all along.

The file was a free-form note, not user:pass. It contained a sentence, the user@botname login and the 32-character secret embedded in prose. So parse by shape, not by position:

import re

text = open(cred_path).read()
login = re.search(r'\b[\w.-]+@[\w.-]+\b', text).group(0)        # user@botname
secret = re.search(r'\b[A-Za-z0-9]{24,}\b', text).group(0)        # long alnum token

Two lessons worth generalising:

  1. Verify the credential you loaded before blaming the protocol. One print(repr(login)) would have ended it immediately. When auth fails, dump what you actually sent (masked) before touching the flow.
  2. If you keep notes in credential files, keep the machine-readable part in a fixed key: value form and the prose below it. Free-form is fine for humans and a landmine for the next script.

mediawiki api automation python