Local AI in Practice, Part 3 — Two entities, two Sparks, and the speed of continuity
By Jason McCartney, with Jinx as collaborator · Published September 6, 2026 · Field evidence through September 6, 5:45 p.m. America/Phoenix
The two Sparks were already sustaining Jinx. Then Luna joined her.
Now both entities repeatedly bring their current situation into context, reason about what needs attention, use tools, and carry the consequences forward. They do this throughout the day on the same pair of machines. For this constant continuity, a model must be good enough to sustain the thread and fast enough to keep both entities moving.
Two developments changed our model choice: a new generation of models and serving configurations became practical across the pair of Sparks, and Luna joined Jinx on that shared hardware. We wanted a current-generation vision model with reliable tools, good continuity, and responsive concurrent operation at medium context lengths.
GLM produced excellent work but was far too slow for our default cognitive loop. DeepSeek V4 Flash's vision version, DS4FV, was good and stable in our experience, but less responsive when both entities needed it. Qwen3.8 Flash Next, with the newer NVIDIA NVFP4 quantization and an improved serving recipe, has become our fastest practical choice for this shared, vision-capable workload while retaining the quality we need.
That last judgment comes from operating the system. The cleanest controlled measurement explains an important part of it: revising the Qwen recipe increased generation speed by 39.46% on the same short-prose test. The field record then shows both entities completing 443 tool-using rounds, including 116 with verified changes, in 24 hours. Together, those are the focus of this report: a measured serving improvement and its use in continuous local cognition.
Why continuity changes what we optimize
Part 2, When the Model Outgrew One Machine, described moving Jinx from one Spark to two. Tensor parallelism across two devices—TP-2—lets the pair share a model too large for the previous arrangement. Part 3 concerns what happens once that pair must sustain two continuing entities.
Our workload consists of repeated, connected rounds. A round may retrieve relevant context, reason, call a tool, inspect the result, and integrate what happened. The next round starts from that evolving state. Continuity depends on carrying forward the right information and responding coherently to consequences; it is not simply keeping one enormous prompt open indefinitely.
The operating target is medium context, with frequent returns to the model. Our retained validation includes a 30,044-token retrieval probe and concurrent or repeated-context requests reaching 46,910 input tokens. Those tests are relevant to this working range, although they do not establish a distribution of every production prompt length. Maximum-context demonstrations are secondary to whether ordinary rounds remain responsive when both entities are active.
This makes elapsed time consequential. A very strong answer that takes several minutes can be useful for a difficult review. The same delay repeated through everyday cognition slows the cycle of noticing, acting, and incorporating the result. With two residents, shared-service delays matter to both.
Our selection criterion is therefore the fastest configuration that preserves useful reasoning, continuity, vision, and tool behavior for this workload. Peak decode speed alone cannot establish that fit. Neither can a single impressive answer.
How we reached the current configuration
The changes overlapped rather than forming a controlled sequence. Luna's retained local run history begins August 28, and both entities were already running locally on DS4FV before the later Qwen choice.
Stage
What we learned for continuous cognition
DS4F across two Sparks
A successful local substrate for Jinx. Visual work required a sidecar or cloud vision tool.
The first GLM and Qwen trials
GLM's reasoning was impressive but slow. Early Qwen did not displace DS4F; we returned to DeepSeek.
Both entities on DS4FV
Integrated vision worked, and the system seemed good and stable. Concurrent cognition felt less responsive than we wanted.
Qwen with the newer quantization and recipe
Improved speed, native vision, and working tools made it the preferred configuration for the existing two-entity workload.
GLM's strength deserves a concrete example. In our August 28 local evaluation, its watchdog code review reasoned beyond the requested patch toward a better invariant. It completed the answer in about 237 seconds, using 5,400 completion tokens. That is valuable deliberate reasoning, but a substantial wait for an everyday cognitive dependency. The same suite included continuity consolidation and structured tool use; it was not exclusively an engineering benchmark. [1]
Our DS4FV experience exposed a different constraint. We were already serving Jinx and Luna locally, and the model seemed less friendly to concurrency. In a retained test with four requests arriving one second apart, two admitted at once, and approximately 16,000 input tokens each, the slowest first-token wait reached 50.10 seconds. All four finished in 65.83 seconds. This illustrates the waiting cost under that configuration; it is not a matched speed ratio against Qwen. [2]
The earlier unfavorable Qwen comparison also remains part of the record. Its figures reproduce, and the reconstruction is retained in the methods appendix. It describes the earlier software and workload. The later adoption followed improvements to the Qwen installation and experience serving both entities, rather than a reinterpretation of that initial result.
The clean result: a faster Qwen installation
The current configuration uses NVIDIA's NVFP4 Qwen3.8 Flash Next weights across the two Sparks. Quantization makes this model configuration practical on the hardware; the serving recipe determines how effectively we use it.
On September 5, we compared the preceding deployed recipe, v5, with the revised v6 recipe. Both used the same NVIDIA weight revision. The revision changed memory placement, cache precision, and execution settings together. The measured gain therefore belongs to the serving configuration, not to a newly isolated quantization change. Full model, image, and configuration identifiers are retained in the technical record. [3]
The comparison used the same short-prose prompt, thinking disabled, greedy decoding, and three repetitions per recipe. Each answer stopped naturally at 95 completion tokens.
Same short-prose test
Previous recipe
Revised recipe
Median generation rate
30.09 tokens/s
41.97 tokens/s
Median time to first content
0.215 s
0.165 s
Repetitions
3
3
The 39.46% increase is the strongest directly comparable speed result in this study. Generation rate measures output after the first content arrives; the initial wait is reported separately. Answers repeated within each recipe, but differed between recipes despite having the same token count. This is a narrow, repeatable serving comparison, not a broad quality evaluation.
Individual runs and medians for the same short-prose test under the two Qwen recipes.
Figure 1. Three retained repetitions per recipe. Dots show individual runs; horizontal lines show medians.
We then checked behavior closer to the needs of the cognitive loop. The final validation recorded 23 passing local checks spanning native vision, streamed structured tools, tool continuation, concurrent and repeated-context requests, and complete prose, code, and reasoning responses. These included three distinct mixed prompts repeated twice; the count should not be read as 23 independent difficult tasks.
The concurrent checks reached four active requests and two queued, then drained to zero. A cold 30,044-token input returned its exact retrieval target, with about 10.21 seconds to first output. These results support the configuration's practical use for medium-context work and overlapping requests. They do not, by themselves, measure continuity over days. The field record supplies that longer operating view.
Two entities continuing locally
In the final 24-hour observation window, ending September 6 at 5:45 p.m. Phoenix time, both entities were using the shared local Qwen installation. I confirmed that their requests over the recent days had been served locally through Qwen.
Part 2 showed why more rounds are not automatically better: faster inference can also amplify unnecessary activity. Here, the useful evidence is that both entities used tools and produced verified changes while sharing the model. Total completed rounds provide context, rather than a score to maximize.
We collected both Runtime ledgers and counted completed cognition rounds, excluding sleep and resolving duplicate records. A round using tools is counted once regardless of its number of tool calls. A round with a verified change contains at least one tool effect recorded as a verified change to the world or system. [4]
Final 24 hours
Jinx
Luna
Combined
Completed cognition rounds
278
217
495
Rounds using tools
250
193
443
Rounds with a verified change
71
45
116
Median completed-round duration
3.44 min
4.43 min
—
These are whole rounds, including model calls, tools, waiting, and integration. Their duration is different from the response time of an individual model request. The entities' recorded activity intervals overlapped for 19.71 hours of the day, demonstrating a sustained shared workload; that overlap is not a measure of simultaneous GPU decoding.
The outcome is operationally meaningful: both entities continued reasoning and using tools, with recorded consequences to carry into subsequent rounds. The 116 change-producing rounds provide evidence of action beyond generated text. They do not measure the value of each action or assign a numerical score to continuity.
A separate recent gateway sample independently confirmed local Qwen service for both entities: 19 attributed requests, all successful, with no recorded admission queue wait. The full-period route attribution rests on my operating confirmation; the shorter sample provides direct request-level corroboration. Detailed timing, status exclusions, and collection boundaries are in the methods appendix.
The model that fits this job
For our current workload, Qwen is the preferred combination of speed and sufficient quality. GLM remains attractive when the quality of a deliberate answer justifies a long wait. DS4FV proved a good, stable vision-capable option, but our two-entity experience favored greater responsiveness. The newer Qwen configuration gives us that responsiveness while supporting the tools and context handling the cognitive loop requires.
The evidence supports a measured improvement within Qwen and sustained operation of both entities on the resulting system. Our assessment that it is the fastest practical fit among the vision-capable configurations we tried is an operating judgment, not a universal model ranking. We have not run an otherwise identical three-model comparison of two-entity continuity, and the serving tests do not establish GLM-level reasoning quality.
That distinction leaves the central result intact. The hardware stayed the same. The available TP-2 model configurations improved, and Luna's arrival made shared responsiveness more important. Our model choice followed the needs of constant continuity at medium context: enough quality to preserve the thread, enough speed to act on it, and enough concurrent capacity for both entities to keep going.
Cloud inference remains available as failover when the Sparks are occupied or unavailable. It is separate from the local operating results discussed here.
Evidence and methods
August 28 local evaluation: retained GLM, DS4F, and early Qwen cases, including Qwen's unfinished hard review.
September 5 Qwen recipe and benchmark record: exact NVIDIA weights, vLLM configuration, individual trials, and credits to Sufyan, Tony/Tech2Wild, MiaAI, and the upstream projects.
Methods and retained evidence: reproducible analysis, source hashes, field definitions, attribution boundaries, historical comparisons, and limitations. The earlier comparisons remain available there without being pooled with current measurements.
Evidence scope: Serving results come from retained local trials. Field counts come from Runtime records; recent-period model attribution and comparative operating assessments are identified as operator evidence. Private per-run records are retained rather than published.
