Insyra v0.3.4 - JSON Acceleration and Data Pipeline Fixes

Insyra v0.3.4 was published on 28 September 2026, delivering performance gains across JSON processing alongside data protection fixes in table export routines. This release is a stable general release rather than an unstable prerelease, targeting production pipelines that ingest delimited files and exchange tabular datasets. The headline improvement integrates an optimized JSON engine while correcting a silent data loss failure when appending CSV records into Excel spreadsheets.

The full release notes and downloads are on the GitHub release page.

Pipelines combining tabular formats often use csvxl to aggregate raw records into multi sheet workbooks. In version v0.3.3, AppendCsvToExcel contained a critical defect: if reading a source CSV failed, the routine emptied the target worksheet and saved the truncated file.

Version v0.3.4 resolves this failure by reading the CSV input entirely before modifying the target workbook. If reading fails, the operation leaves the target sheet untouched.

The fix changes sheet replacement semantics. AppendCsvToExcel now overwrites the existing sheet with a fresh worksheet to guarantee data reads back correctly. However, previous formatting attributes, such as column widths, views, and merged ranges, are not preserved. Upstream issue #401 tracks preserving those formatting options.

Insyra now routes all JSON operations through github.com/goccy/go-json version v0.11 instead of the Go standard library.

Benchmarks show lower execution latency. On a 100000 row, five column table producing a 14 MB payload, ToJSON_Bytes dropped from 66 ms to 45 ms, while ReadJSON decreased from 129 ms to 90 ms on Apple M3 hardware.

The update introduces breaking parser behavior:

  • Numbers with leading zeros, such as {"a":01}, now produce syntax errors instead of parsing as 1.
  • Numbers exceeding float64 limits, such as 1e400, parse as raw strings rather than aborting the entire document read.
  • Invalid UTF-8 bytes inside strings become U+FFFD replacement characters.
  • Indented JSON serialization in ToJSON, ToJSON_Bytes, and ToJSON_String mirrors encoding/json.MarshalIndent, outputting small exponents as 1e-7 rather than 1e-07.

Command line workflows frequently chain operations using workspace state saved in state.json. Earlier versions mutated data structures across commands: DataTable instances lost column ordering, column letters shifted, and types degraded, converting float64 values like 3.0 into int64 and dropping duration types.

Version v0.3.4 stores native Go types directly in state.json:

  • DataTable variables preserve original column order, column names, row names, and cell types across commands. Running insyra cols t after insyra load c.csv as t lists columns in file order rather than alphabetical order.
  • Date fields processed via parsedates remain dates for subsequent resample commands.
  • Time intervals calculated in Common Calculation Language expressions stay time.Duration types.
  • Fitted scalers from scale fit and clustering trees from hclust persist across isolated commands, allowing subsequent scale transform and cutree calls to succeed.
  • Workspace imports via insyra env import no longer overwrite non empty targets without --force if configuration files cannot be read.

Errors that carry another error as their cause now wrap it with %w instead of formatting it into text. Callers can now inspect failures programmatically using errors.Is and errors.As. This wrapping covers CSV encoding detection, Excel sheet reads, GLPK solver loading in lp, pd Series creation, and CCL functions including MID, SUBSTR, TONUM, VALUE, TOSTR, and TEXT.

Preprocessing scalers, including StandardScaler, MinMaxScaler, RobustScaler, and MaxAbsScaler, implement json.Marshaler and json.Unmarshaler. Previously, passing a fitted scaler to json.Marshal returned empty {} objects. Fitted parameters, column letter bindings, and NaN states now round trip cleanly through JSON.

Additional core updates include:

  • Lookup operations in Count, Counter, Find, and Replace match nested arrays and nested slices holding identical elements across all sizes.
  • Machine learning autograd additions include Tape.Custom and Tape.BackwardFrom, resolving issue #375 by connecting external operations with explicit vector Jacobian products.
  • Statistical routines, including paired, rank, and ANOVA tests, refuse NaN and infinite values.

Upgrading to v0.3.4 maintains Go 1.25.12 as the minimum compiler baseline. Operators should review two breaking changes:

  1. ReadJSON rejects non compliant JSON syntax, including numbers with leading zeros.
  2. AppendCsvToExcel overwrites existing sheets and discards merged cell ranges and custom column widths.

Existing state.json files remain compatible and are automatically rewritten into the new typed layout upon save. Dependencies were updated to current patches, with grpc v1.84.0 addressing security advisory GHSA-2v4p-qf9q-27wj.