I fine-tuned GLM 5.3 Flash on Fireworks to count doors, windows and sinks on real floor plans. It went from 65.4% to 78.4% (+13.0 points), and two statistical tests agree the gain is real. This page walks through how I got there in six steps, so you can run the same playbook on your own task tomorrow.

Everything here (data prep, notebooks, results) is in ai-experiments on the blueprint-experiments branch. floorplancad-symbol-counting/ is the project below; eng-drawings-extraction/ is a second complete example you can think of the same way.

Disclosure, the way the rules ask for it: I built this before the event, as a sample. The rules (section 4) require you to disclose pre-existing work and material use of AI coding assistants. I used a Cursor agent throughout, pointing it at both the official Fireworks cookbook and my own experiments github.

Step 1: Find a task worth doing

I picked FloorPlanCAD: counting symbols on real architectural floor plans, a job estimators do by hand before every construction bid.

I asked my Cursor agent to browse Hugging Face for vision datasets with real labels and a clear right answer. It found two, so I wrote a short business case for each before choosing.

Why FloorPlanCAD won: it comes from a published paper, FloorPlanCAD (Fan et al., ICCV 2021), with real drawings, so a judge can trust the labels and the task. The engineering drawings project stays in the repo as a second worked example. Its fine-tune gained more (79.8% to 95.4% JSON F1, +15.6), but on synthetic drawings and a different metric.

Size the value with the assumptions written out. Saving an estimator 2 hours per bid on 50 bids a year is about $3,800 in wages, at the BLS median of $37.86 an hour. Replace those assumptions with numbers from a real estimator if you can.

Check your labels before you trust your eval. The Hugging Face copy of FloorPlanCAD has scrambled class names: sinks are labeled wardrobe, toilets escalator. I rebuilt every label from the official SVG files in 01_official_labels.ipynb. Then I turned each floor plan tile into two questions with one right answer each: "How many doors are shown?" (1 to 12) and "Does this plan contain any sinks?" (yes or no, balanced about 50/50).

Step 2: Run a baseline on an open model

Base GLM 5.3 Flash scored 65.4% on 1,000 held-out questions, for $0.23 total on Fireworks serverless. Everything after this has to beat that number.

Lock the test set first. I split by building, 80/20, so no building appears in both training and test. The test set is 1,000 questions from 185 buildings (495 counting, 505 yes/no).

Save one row per question, with the building it came from. Both tests in step 6 need these rows, and the building id matters later:

{"id": "0612-0238-count-door", "building": "0612", "answer": 5, "pred": 2, "score": 0.0}

Keep the baseline fair. Same prompt and settings for every system you compare, temperature 0. Weakening the baseline to make your gain look bigger is against the rules.

The call is the standard OpenAI client pointed at Fireworks, so swapping models is a one-string change:

client = AsyncOpenAI(base_url="https://api.fireworks.ai/inference/v1",
                     api_key=os.environ["FIREWORKS_API_KEY"])
resp = await client.chat.completions.create(
    model="accounts/fireworks/models/glm-5p3-flash",
    messages=messages(r), temperature=0.0, max_tokens=4096)

I also ran Gemini 4 Argon, Claude Opus 5.5 and GPT-6.1 Terra Pro on the same questions (70.5–76.8%). That's optional. The required comparison is your open model, before vs after.