Omni-macos is presented, which runs its encoder, index and store on the Mac that already holds the files, so no indexed file, no typed query and no vector ever leaves the machine.
Abstract
A search engine that embeds text, code, documents, images, audio and video into the same representation space has to run its encoder and keep its index somewhere, and almost every component built for the purpose assumes a server. We present omni-macos, which runs its encoder, index and store on the Mac that already holds the files, so no indexed file, no typed query and no vector ever leaves the machine. It keeps a background indexer and an interactive search box inside one memory budget the user sets: it re-encodes only the chunks an edit changes, hands the GPU smaller units while the user is typing, answers queries from a one-bit replica of the index with exact rescoring, and propagates that budget to the allocators that draw on unified memory. We measure on five Macs spanning an eightfold range of accelerator width and a thirty-twofold range of memory, each indexing the files it already holds.
OptGear-1M is the first generative language model to achieve 20 TPS with W4A32 quantization on the ARM Cortex-M7 CPU of the STM32H747I-DISCO, and is the most data-efficient of the existing foundation models.
A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.24031). This report keeps the hardware and asks what a model that fits can do: we deploy MiniCPM-V-4.6, a modern multimodal assistant pairing a SigLIP2 vision encoder and window-attention merger (16x visual token compression) with a compact hybrid gated-delta-net backbone, entirely on the GPU. Three results. (i) An all-GPU engine built on measured foundations: projections that dequantize 8-bit weights once and call the vendor SGEMM still in the last Fermi toolchain (64% of FP32 peak; our best hand-written GEMM hit 37%, wrongly called the ceiling); a chunked delta-rule rewrite of the recurrent layers, 2.8x faster than the sequential scan once attribution exposed one bad kernel; and a measured negative: 4-bit weights make decode slower than 8-bit here, since Fermi issues nibble-unpacking shifts at half rate. (ii) The vision side is a port with a proof obligation: we translate tower, merger, and projector to sm_20 CUDA, validating every stage against a locally generated reference forward (full tower 1.4e-5). One failure, position-embedding bucketization differing on exact rational ties, generalizes to a rule: float tie-breaking in index arithmetic is implementation-defined; call the reference operator, do not reimplement it. (iii) Long context exposes an O(N^2) wall short benchmarks hide: prefill falls from 114 tok/s at 2k tokens to 21 at 10k in a naive attention kernel; per-head vendor-GEMM calls writing into the existing score buffer (zero extra memory) restore a flat profile (408 at 2k, 361 at 10k; 17x), verified by exact needle retrieval from 60% depth. The same rewrite cuts image encoding 6x, to 0.93s. The system answers an image question end-to-end in 1.7s.
Across three decoder architectures on an H100 the authors measure that parse, not copy, holds 64-72% of device-resident decode time; that bounding back-reference chain depth - provable, and costing 0.006% in ratio - moves latency by at most 2.8% and, for the file's own latency spike, provably by nothing at all.
Open-source Speech-to-Text (STT) systems are increasingly deployed across a wide range of hardware, from GPU servers to resource-constrained edge devices. However, these systems are rarely evaluated jointly across heterogeneous platforms with respect to accuracy, latency, and resource usage, making deployment decisions difficult in practice.We present a cross-hardware benchmark of six open-source STT engines (Whisper, whisper.cpp, Sherpa-ONNX, Silero, Vosk, Pocketsphinx) across four platforms—Linux GPU server, macOS, Windows, and Raspberry Pi 5. Using a unified and reproducible evaluation harness, we measure accuracy (WER/CER), latency and throughput, and CPU/GPU/memory usage on clean, noisy, and multilingual datasets at two scales (1.5k and 10k audio clips).Our results show that STT performance is deployment-dependent rather than universal. Sherpa-ONNX yields the lowest error rates on desktop platforms, Silero provides the lowest CPU latency for interactive use, and Whisper delivers the highest throughput under GPU acceleration. On Raspberry Pi 5, only lightweight models achieve near-interactive performance, exposing a fundamental accuracy–latency trade-off in edge environments. We further observe that robustness to noise does not necessarily correlate with clean-speech accuracy, highlighting the importance of evaluation under realistic conditions.This work provides actionable deployment guidance and a reproducible benchmark that surfaces system-level trade-offs in open-source STT across the edge–cloud continuum. Scripts, model configurations, and environment manifests are provided to support replication and extension on new hardware.
Uyen Thy Duong, Pakeeza Akram· International Conference on...· 0 citations
The first complete description of SweepLSD is given, a line segment detector that reads the image exactly once and emits each segment within a few rows of its last pixel passing the scan line, with the tightest frame-time distribution and the best per-segment direction accuracy of the four detectors.
This work studies how to compress a tokenizer by distillation, taking as the supervision target neither the discrete token nor the output distribution but the pre-quantizer latent the model actually consumes - the last representation the two token interfaces share.
Prasanth Yadla, Mohammad Samragh, Dongseong Hwang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
