savoji/qwen3.5-0.8b-tablevqa-lora
Updated • 17
Sajjad Pakdamansavoji
savoji
AI & ML interests
VLM, VLA, LLM, Robotics, Computer Vision
Recent Activity
posted an update about 2 hours ago
Trajectory2Gym raises five Qwen3 models (1.7B to 30B-A3B) by 16.9 to 29.8 points on the equal-weight mean of Tau1, Tau2 and BFCL. Plain SFT, no RL.
A successful tool-use trajectory is not automatically good supervision. In a pool of 34,772 certified candidates, 40.9% contain at most two supervised decisions and only 10.0% have a certified decision after an authentic error.
So the method builds interactions around under-represented multi-step dependencies. An action becomes a target only if it executed validly and is supported by what the agent could see at that step (g_t = e_t v_t). Failed calls stay in context with no imitation loss; the grounded recovery that follows becomes the target.
📊 The numbers
• Post-trained 4B: 69.17 on Tau2, vs 28.33 for the untrained 30B-A3B
• Post-trained 30B-A3B: 84.17 on Tau2, vs reported 84.76 for GPT-5.2 (high) and 85.31 for Claude-4.5-Opus (high)
• EnvFactory-8B nearly ties the 8B model on BFCL (77.55 vs 78.62) but trails it by 47.5 points on Tau2
• Biggest ablation gain (+8.04 mean): parent normalization, weighting each interaction equally instead of each rendered row (Tau2: 52.50 to 74.17)
⚠️ Caveats
• Ablations are single, non-additive runs on Qwen3-8B
• BFCL Live Simple drops 6 to 13 points at four of five scales
• The code alone does not reproduce the reported numbers; the public pool is not the per-scale training sets
🛠️ Try it
• The Apache-2.0 code, five checkpoints and the 34,772-interaction pool are public
https://huggingface.co/papers/2610.33568
https://github.com/yihaohu0118/Trajectory2Gym
In your agent SFT data, how many imitated actions could the agent justify from what it saw?
• Medium: https://medium.com/p/ce1123e4e18b
• Substack: https://sajjadpakdamansavoji.substack.com/p/trajectory2gym-a-4b-qwen3-beats-the
• X: https://x.com/saavoji/status/2115470777499656285
• Website: https://sajjadpsavoji.github.io/blog#trajectory2gym posted an update 1 day ago
GPT-6.1 Terra scores 6 points higher than GPT-5.6 Terra on the Artificial Analysis Intelligence Index, at $0.40 per task instead of $1.40.
Per-token prices fell to $1/$5 per million input/output tokens, from $2/$12. For agent and computer-use workloads, cost stops being the blocker. The question becomes whether the model is reliable enough to act on its own.
The biggest reliability change: on AA-Omniscience it guesses 38% of the time when it does not know the answer, down from 88%, and accuracy rose 2 points.
📊 The numbers (Artificial Analysis)
• Low and medium effort are Pareto-optimal for cost; at lower effort the index gain is 7 to 13 points
• Terminal-Bench 4.0: about +20 points, rivaling GPT-6.1 Sol at max effort
• Coding Agent Index v2.0: 48, vs 54 for GPT-6.1 Sol and 31 for GPT-6 Luna
• Cached input $0.05 vs $0.20, cache write $1.25 per million tokens
⚠️ Caveats
• OpenAI says it matches GPT-6 Astra on Agents' Last Exam at one tenth the price per token and finishes tasks over 3x faster, but gives no absolute scores and there is no independent check yet
• Most other evals gain little, for example 5 points on GDP.pdf
• 38% guessing is still high for an agent that takes actions
• $0.40 per task is measured on AA's index, and your token usage will differ
🛠️ Try it
• API model name: gpt-6.1-terra
• Rerun your own agent evals at low and medium effort and compare task success and error rates, not price
https://openai.com/index/introducing-gpt-6-1-terra/
https://x.com/ArtificialAnlys/status/2114781865811063259
If you run computer-use agents on GPT-5.6 Terra, what error rate would you need to see before switching?
• Medium: https://medium.com/p/2178290c2b22
• Substack: https://sajjadpakdamansavoji.substack.com/p/gpt-61-terra-settles-the-cost-question
• X: https://x.com/saavoji/status/2115105385388683774
• Website: https://sajjadpsavoji.github.io/blog#openais-gpt-6 posted an update 2 days ago
NVIDIA's GEAR lab says its open Flora models give Astra-level robotics data labeling at 125x lower cost. The models already label 200K hours of video a day on 500 GPUs.
Flora is not a robot policy. It is a suite of open perception models that watch raw human and robot video. They label what happened (timestamped instructions) and how well it went (rewards and failure reasons). Yunze Man says the team uses Flora to train NVIDIA GR00T, so this labeling layer helps build the policy.
The weights are public, so the data engine is no longer a private advantage. Anyone with footage can build the same curation, reward and evaluation loop on it.
🔍 What Flora outputs
• Timestamped captions to curate and filter human egocentric and robot data
• Dense progress rewards for quality control and RL post-training
• Rubric scores for real-world benchmarking
• Failure reasons, so failed attempts become usable training data
📊 The numbers
• 125x lower cost at "Astra-level performance" (claimed)
• 200K hours of video labeled per day on 500 GPUs, in production at NVIDIA
• Works on human footage, which is cheaper to collect than robot data
⚠️ Caveats
• No benchmark, metric or cost breakdown supports the 125x figure, and Astra is never described
• The announcement gives no GPU type, model sizes, license or input specs
• Learned rewards could add bias or reward hacking to RL, and no accuracy numbers are published
• Every claim comes from the team that built it, so test it on your own data first
https://x.com/DrJimFan/status/2112597587987689912
If you train VLAs on egocentric video, which signal would you plug in first: captions for filtering, or progress rewards for RL?
• Medium: https://medium.com/p/2fd29045dffb
• Substack: https://sajjadpakdamansavoji.substack.com/p/nvidias-flora-puts-the-robot-data
• X: https://x.com/saavoji/status/2114758981474509137
• Website: https://sajjadpsavoji.github.io/blog#nvidia-just-openedOrganizations
None yet