ArcaLunar’s Blog

A running blog of notes across ACM, AI/ML systems, math, and everything in between.

• Quantum Factorization Algorithm 2026-10-04 quantum
• Circuit Complexity 2026-10-02 quantum Discuss the concept of circuit complexity, the number of elementary operations used to compile a circuit.
• Proof of Universal Gate Set for Any Quantum Computation 2026-10-02 quantum A proof of the claim that CNOT + single qubit gates are universal.
• Universal Quantum Computing 2026-10-02 quantum Discuss the key ideas in the proof of universality; universal set of quantum gates; fidelity.
• Linear Algebra for Quantum Computing 2026-10-01 linalgquantum Linear algebra guide for quantum computing, based on supplementary material of HKU COMP3366.
• Overview of Recursive Self-Improving (RSI) 2026-09-30 airsi RSI refers to the procedure where an AI system analyzes, updates, and redesigns itself to enhance next-generation’s capability and efficiency. It creates a feedback loop where each generation performs better than previous generation. But personally speaking, RSI is … probably another hype.
• Cross-Tokenizer Challenges in RL 2026-09-30 rl In RL, when the reference model and to-be-trained model have different tokenizers, training with RL algorithms will encounter some problems. Such problems are usually referred to as “cross-tokenizer” (cross-vocabulary) problems.
• RL Training-Inference Mismatch, Diagnostics, and Solution 2026-09-30 rl Training-Inference Mismatch refers to a class of issues that the logprob of tokens mismatch between inference stage and training stage, causing training problems like reward dropping, etc. Results could be amplified in case of MoE training. This post briefly discusses related problems, diagnostics, and possible causes.
• Closure in Scheme 2026-09-30 scheme Introduces how closure works in Scheme programming language.
• Fully Detach a Process into the Background without tmux or Similar Tools 2026-09-30 linux Introduces nohup (no hangup) and the & operator to move a process into the background so that it keeps running even after the terminal is closed.
• SQL 2026-09-29 dbms
• Launch Options in verl-omni Example Scripts 2026-09-29 verl_omni Briefly introduces the launch options in verl-omni’s example scripts, as some of them are related to issues like OOM and algorithm tuning. This post first takes LTX2.3 FlowGRPO training script as an example.
• Latent Diffusion Models 2026-09-28 diffusionlatent Performing diffusion in latent space can make diffusion much more efficient in terms of generation and training.
• LTX2: Efficient Joint Audio-Visual Foundation Model 2026-09-27 foundationtextaudioimagevideo LTX2 is a foundation model for text-to-audio-video (T2AV) and text-image-to-audio-video (TI2AV) with the input image being the first frame of video.
• DSec: DeepSeek Elastic Compute 2026-09-24 deepseekagent DSec is a scalable agentic training system for LLMs. Agentic training requires cooperation among various components, including OS, software, network, proxy, etc., as well as security issues, e.g., escaping, dangerous hacking. DSec is an open-source solution for this.
• Chat Template of LLM with Jinja2 2026-09-24 llm A chat_template.jinja is to format messages, tools, and responses into a prompt sent to the LLM. It uses the Jinja2, a Python packaging widely used in Flask, Huggingface, etc.
• Entity-Relationship Model 2026-09-24 dbms The Entity-Relationship Model in Database Management System.
• An Introduction to DBMS 2026-09-24 dbms Data Abstraction, Data Modeling, Database Languages in Database Management Systems (DBMS).
• Analysis of Workflow of MainDiffusionV1 in verl-omni 2026-09-24 verl_omni Analysis of the main_diffusion_v1 entrance of v1 trainer.
• Composite Quantum Systems 2026-09-24 quantum Like bits, qubits can be put together for more computational power.
• Full-Vocabulary, Top-k, and Sampled Token Distillation 2026-09-23 distillation This post briefly discusses the difference between full-vocabulary, top-k, and sampled-token distillation.
• Safetensors and GGUF Format 2026-09-23 Safetensors and GGUF have become the de facto data format to efficiently store model weights in a readable way. This post briefly analyzes their format protocols.
• Set Default Values for Variables in Bash Scripts 2026-09-23 bash This post introduces some useful snippets and grammars in Bash shell for scripting to set default values.
• Qubit Gates 2026-09-23 quantum We can actually process quantum information using qubits.
• Rethinking OPD II 2026-09-22 opdrl The work following Rething OPD. It also investigates the mechanism of OPD, but this time, from data’s point of view.
• Rethinking OPD 2026-09-22 opdrl This work provides an interesting point of view on analyzing LLM On-Policy Distillation. Probably migratable to multimodal RL.
• Unification of Flow Matching and Diffusion 2026-09-22 diffusionflow_matching Flow Matching is an ODE-based method, while Diffusion is an SDE-based method. This means that some RLr algoithms designed for Diffusion cannot be applied to FM. This post briefly documents how to formulate FM with SDE, therefore unifying Diffusion and Flow Matching.
• The Workflow of Training with verl-omni main-diffusion Pipeline 2026-09-22 This post is to review the overall workflow after launching a task from terminal bash examples/..../xxx.sh. We take FlowGRPO+SD3.5 Medium as an example here.
• Training Stable Diffusion 3.5 Medium with DiffusionOPD on OCR Tasks 2026-09-21 verl_omniDiffusionOPD I ran the experiment of post-training SD3.5 Medium with DiffusionOPD on OCR tasks, as documented in the docs. I post some results and figures here for my reference.
• Replay Buffer in verl 2026-09-21 verl Briefly analysis the ReplayBuffer class design in upstream verl and downstream verl-omni.
• Thoughts about the JEV Model 2026-09-21 thoughts The JEV has gone viral for a weird reason of cheap and fast, along with its “Reinforcement Learning for Calibration Decision” (RLCD). I want to comment some my questions and thoughts here.
• Skip wandb SSL Verification 2026-09-21 techtipswandbssl wandb is a useful platform for auto recording information and visualizing for deep learning projects. This post documents how to sync wandb logs in brief.
• How is DiffusionOPD Implemented in verl-omni 2026-09-20 DiffusionOPDverl_omni This post is more of coding aspect rather than of mathematical aspect. This post briefly discusses how DiffusionOPD is implemented in verl-omni.
• Preparing OPD Teacher by Merging LoRA weights in verl-omni 2026-09-18 verl_omni Follow the official tutorial for DiffusionOPD Trainer. This post briefly describes how to merge LoRA weights into base model through Diffusers.
• Metrics Recorded by verl-omni 2026-09-18 verl_omni This post briefly reviews all the metrics recorded by verl-omni during training. The metrics can be roughly categorized into several classes, hardware performance, time consumption, critic, actor, rollout correction, training, etc.
• Overview of Ray 2026-09-17 ray Ray is the underlying runtime for modern infrastructure like vllm, verl, etc. An attempt to understand how Ray works :)
• The v1 and v0 Trainer in verl-omni 2026-09-17 verl_omni Briefly introduce the v1 and v0 RL trainer in verl-omni. Instead, v0, v1 (sync/async) share the same underlying engine (vllm-omni for rollout engine and FSDP/VeOmni for actor engine), but differ in dataflow and scheduler engine.
• Hydra 2026-09-17 hydra Hydra is a Python framework for configuring complex applications.
• Roadmap of Generative Models 2026-09-14 roadmap A roadmap of understanding modern generative models, including VAE, GAN, diffusion, and flow matching.
• Roadmap of Reinforcement Learning 2026-09-14 rlroadmap A roadmap of study reinforcement learning. Posts are listed from basics to modern works.
• DPO: Direct Preference Optimization 2026-09-14 rl DPO is an optimization that simplifies the two-stage pipeline containing reward model training in RLHF and PPO post-training into a single supervised loss function.
• Generative Models through Normalizing Flows 2026-09-14 generative Normalizing flow (NF) refers to a family of generative models. Its paradigm is to transform a simple distribution into a complex distribution through a series of inversible functions. Therefore, we’re able to compute the likelihood and apply maximum likelihood learning. Generation is done through sampling from the simple distribution and transforming it with the series of function.
• Policy Gradient Theorem 2026-09-14 rl Policy Gradient Theorem is an important topic in RL as it envisions many popular policy-gradient RL methods, including REINFORCE, Actor-Critic, A2C, PPO, GRPO, etc.
• Fundamental Concepts in Markov Decision Process and Reinforcement Learning 2026-09-13 rlMDP This post roughtly covers (1) the setups in MDP/RL, (2) value function, (3) Q-state, (4) discounting, (5) advantage, (6) Bellman Equation (7) the difference between on- and off-policy. In addition, this post tries to provide an intuition beyond formula.
• Crash Course of Scheme-lang Grammar 2026-09-13 plscheme The let family in Scheme programming language. They are used to bind values to variables.
• Advantage Actor-Critic 2026-09-12 rl Advantage actor-critic (A2C) is an improvement of actor-critic. The introduce of advantage further stablizes the learning procedure of the critic model.
• PPO: Proximal Policy Optimization 2026-09-12 rl The idea of PPO can be summarized as: allow the policy to update, but constrain the policy from changing too much, in order to stablize training. Compared to prior work TRPO that uses second-order methods which requires massive computation, PPO uses first-order methods to achieve the same goal.
• TRPO: Trust Region Policy Optimization 2026-09-12 rl
• Self-Evolving of AI 2026-09-11 Self-Evolving has become a core concept in AI fields.
• DiffusionOPD 2026-09-10 diffusionrlopd On-policy distillation for diffusion models.
• OPD: On-Policy Distillation 2026-09-10 rlopd On-policy distillation
• Actor-Critic Methods in Reinforcement Learning 2026-09-08 rl Actor-Critic is a RL method that combines value-based and policy-based methods to help stablize the training by reducing the variance through an actor controlling how agents behaves (policy-based) and a critic judging how well agents behave (value-based).
• GRPO: Group Relative Policy Optimization 2026-09-08 rl GRPO is a method introduced by DeepSeek Math, a variant of PPO that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO.
• Reinforce 2026-09-08 rl Reinforce is a policy-based method, but instead of obtaining policy from Q-states or values, it parameterizes the policy \(\pi:\mathcal{S}\mapsto\mathbb{P}(\mathcal{A})\) that outputs a distribution of actions and trains the policy through gradients.
• A Survey of 16 Open-Source RL Libraries 2026-09-07 rlframeworksurvey As of 2026, among the 16 surveyed open-source RL libraries, the common points are that: (1) inference and training are disaggregated; (2) they are connected by a rollout buffer to temporarily store model outputs; (3) weights are transferred asynchronously. Other findings include: (1) Ray dominates orchestration; (2) NCCL broadcast is the default method for transferring model weights; (3) staleness management, referring to how to handle outdated data samples, ranges from least-recent method to advanced importance-sampling correction; (4) LoRA training is sparsely supported; (5) distributed MoE support is becoming trend. See more at Huggingface Blog
• LLaDA: Large Language Diffusion Models 2026-09-06 generativediffusionllm LLaDA challenges the traditional autoregressive next-token prediction LLMs. It introduces LLaDA, a diffusion-based language model with masking. In short, tokens are progressively masked during training, and a bidirectional Transformer learns to predict all masked tokens simultaneously by optimizing a proper likelihood lower bound rather than a cross-entropy objective.
• Loop Transformer 2026-09-06 llm Loop Transformer goes viral recently along with the release of OpenAI’s new models. Compared to traditional way of deepening new models by stacking up more layers, Loop Transformer does so by reusing middle layers cyclically.
• DistFlow: A Fully Distributed RL Framework 2026-09-05 rlinfradistributed DistFlow proposes a fully distributed multi-controller paradigm by decoupling data transfer and control dispatch. For control, DistFlow introduces DAG-based task scheduler for finegrained and independent execution.
• HetRL: Efficient RL in Heterogeneous Environments 2026-09-05 rlinfra HetRL is proposed for efficient RL post-training in compute-heterogeneous environments. HetRL forms the RL infrastructure as a joint optimization problem and proposes 2 scheduling algorithms (near-optimal based, and integer lineaer programming based) for RL infra.
• HeteroRL (GEPO): Group Expectation Policy Optimization 2026-09-05 rl HeteroRL is an architecture to decouple rollout sampling and policy optimization, the core of which is GEPO, an algorithm proposed to alleviate high variance of importance weights and training instability caused by increased KL divergence resulting from high latency due to heterogeneity in compute resources.
• Confidence Interval Estimation 2026-09-04 statistics
• Helpful Resources 2026-09-03 Lectures, Course URLs, Blogs, Papers, etc. useful for my study.
• Permutation Groups 2026-09-01 mathsgroup
• Overview of Convex Optimization 2026-09-01 mathsoptimizationconvex Convex optimization focuses on a subset of optimization problems, i.e., the objectives and constraints are convex, which means \( f(ax+by) \le a f(x) + b f(y) \).
• Qubits and Measurement 2026-09-01 quantum Qubit is the quantum system that we want to study and is in some quantum state. Observable is the physical quantity 物理量 that we want to query about and is represented by Hermitian matrices. Measurement is the actual action that we perform a query and read out a classic outcome.
• Common Metrics 2026-08-31 evaluation Metrics are important in evaluating the quality of LLMs. Although nowadays, domain-specific datasets used for evaluating LLMs mainly treat metrics like task accuracy as the only golden standard, there are still general metrics for evaluation.
• Multi-Head Attention and Grouped-Query Attention 2026-08-31 attention
• ZeRO: Zero Redundancy Optimizer 2026-08-31 trainingdistributed ZeRO adopts different levels of distributed strategy to remove redundancy in memory, at the cost of more communication. This work also gives birth to FSDP, a widely used distributed training schema.
• GDN - Gated Delta Network 2026-08-29 attention
• N-Gram Embedding 2026-08-29 llmembedding
• Cyclic Group 2026-08-29 mathsgroup
• Using jq 2026-08-29 linux
• Solution to Being Unable to Clear Screen 2026-08-28 linux
• A Taste of ExecuTorch 2026-08-26 torchinference ExecuTorch is PyTorch’s cutting-edge solution for on-edge model deployment, with limited support for training.
• FedHera 2026-08-26 federated
• Using adb 2026-08-26 android adb is a useful tool for diagnosing Android phones, as well as running C++ programs on Android phones.
• SigLIP: Sigmoid Loss for Language Image Pretraining 2026-08-25 multimodalvlm The advantage of SigLIP over CLIP is that SigLIP only relies on image-text pairs rather than global pairwise similarity, reducing computational cost from quadric to linear w.r.t. batch size.
• Introduction to Social Psychology 2026-08-25 artspsychologysociology
• Bayesian Asymototics: From Universal Source Coding to General Large-Scale MIMO Capacity 2026-08-19 information_theory
• Thinking with Visual Primitives 2026-08-17 multimodal
• Kimi K3 2026-08-14 llm
• Builder 2026-08-14 swe
• Double Buffering and Pipelining 2026-08-14 gpucuda Double buffering is a pipelining technique in GPU programming. It allocates two staging buffers and alternates between them, so that the data movement for iteration \(k+1\) can overlap computation for iteration \(k\). Pipelining is the genralized version of double buffering that splits tasks into three stages, and arranges stages to make them overlap to hide latency.
• Vectorization 2026-08-14 gpucuda
• Natural Numbers and Integers 2026-08-14 number_theory
• Iterators 2026-08-14 lua
• Results and Error Handling 2026-08-14 lua
• IO and OS 2026-08-12 lua
• String STL 2026-08-12 lua
• Coroutines in Lua 2026-08-12 lua
• OOP Patterns 2026-08-12 lua
• Tables in Lua 2026-08-12 lua
• Logging Library 2026-08-12 python Introduces useful libraries for logging.
• X-Fi: Modality-Invariant Foundation Model 2026-08-11 multimodal
• Numbers in Emacs Lisp 2026-08-11 elisp
• Printed Representation and Read Syntax 2026-08-11 elisp
• Lists 2026-08-11 lispelisp Lists are the most important data type in Emacs Lisp.
• Numbers 2026-08-11 lispelisp Briefly introduces basic operations related to numbers.
• Strings 2026-08-11 lispelisp
• Crash Course of Lua 2026-08-11 lua
• Sample-Efficient Integration of New Modalities into LLMs 2026-08-10 multimodal An interesting work that follows the idea of adopting a projector for modality adaptation, while leveraging a “hypernetwork” for generating LoRA weights from data samples to apply to the projector for adaptation.
• RAID 2026-08-10 os Redundant Array of Inexpensive Disks (RAID) is a technology to build a faster, bigger, and more reliable disk system.
• Project Configuration 2026-08-10 uvpython
• uv: Swiss Army Knife for Python Projects 2026-08-10 tooluvpython
• Free Space Management 2026-08-09 os
• Paging, TLB and Page Tables 2026-08-09 os Introduction to paging mechanism.
• 14th Chongqing CPC 2026-08-07 acm
• Use reflector to Select Best pacman Mirrors 2026-08-06
• Grammars for Building Container Images 2026-08-05 dockerpodman ContainerFile and DockerFile are almost the same, the difference between which is simply that the former is the default for podman, while the latter is the default for docker.
• Ambient Sensing and Intelligence Where AI Meets Physics 2026-08-04 aiseminar
• Main Branches of Buddhism 2026-08-03
• Linear Regression 2026-08-03
• Sample Distribution Theory 2026-08-03
• Overview of Integer Programming 2026-08-03
• Branch-and-Bound Method 2026-08-03
• Eigenvalues and Eigenvectors 2026-08-03
• Jordan Form 2026-08-03
• Orthogonality 2026-08-03
• Determinants 2026-08-03
• 3DGS 2026-08-03
• Principles of Network Applications 2026-08-03
• Video Streaming and CDN 2026-08-03
• socket Programming 2026-08-03
• Email 2026-08-03
• DNS 2026-08-03
• HTTP 2026-08-03
• Decorator 2026-08-03
• Facade 2026-08-03
• Flyweight 2026-08-03
• Proxy 2026-08-03
• Visitor 2026-08-03
• Observer 2026-08-03
• State 2026-08-03
• Strategy 2026-08-03
• Template Method 2026-08-03
• Memento 2026-08-03
• Command 2026-08-03
• Iterator 2026-08-03
• Mediator 2026-08-03
• Prototype 2026-08-03
• Singleton 2026-08-03
• Automatic Memory Management 2026-08-03
• Code Generation 2026-08-03
• Dataflow Analysis 2026-08-03
• IR Optimization 2026-08-03
• Register Allocation 2026-08-03
• Runtime Management 2026-08-03
• Semantic Analysis 2026-08-03
• Raft Consensus Protocol 2026-08-03
• Two-Phase Commit 2026-08-03
• EEVDF 2026-08-03
• Overview 2026-08-03
• Advanced Disjoint Set Union 2026-08-03
• Persistent Segtree 2026-08-03
• Link Cut Tree 2026-08-03
• Splay Tree 2026-08-03
• WBLT 2026-08-03
• Containers in Odin 2026-08-03
• Struct of Array Types 2026-08-03
• Associative Array 2026-08-03
• tokio Tutorial 2026-08-03
• Writing Commands 2026-08-03
• ArchLinux Connect to HKU Campus WiFi 2026-08-03 linux Connect to HKU Campus WiFi using CLI.
• OpenTSLM 2026-08-01 time_seriesllm OpenTSLM is a multimodal model that tries to integrate time series and textual data.
• The BTRFS Filesystem 2026-08-01 btrfs
• Monad Transformers 2026-08-01 haskell
• State Monad and StateT Monad 2026-08-01 haskell
• Rank-N Types 2026-08-01 haskell
• Functions 2026-08-01 koka
• ArchLinux Package Build Script 2026-08-01 pkgbuild AUR is recently under attack, so it becomes important to be able to understand pkgbuild, the script to build packages in Arch Linux.
• The Foldable Typeclass in Haskell 2026-07-31 haskell
• Modeling the Context 2026-07-31 haskell
• Reader 2026-07-31 haskell
• The Traversable Typeclass 2026-07-31 haskell
• Generalized ADT 2026-07-31 haskell
• Vector Spaces 2026-07-28 mathslinalg
• DDPM: Denoising Diffusion Probabilistic Models 2026-07-27 generativediffusion
• DiT: Scalable Diffusion Models with Transformers 2026-07-27 generativediffusion DiT has a relatively simple idea that it replaces the U-Net in usual Latent Diffusion Models (LDMs) with Transformers (ViT).
• Maximum Likelihood Learning 2026-07-27 probabilistic Maximum likelihood learning is a practical approach to train generative models that adopts probabilistic modeling.
• Score-Based Models — Working Notes 2026-07-27 generativediffusion
• Flamingo: A Visual Language Model for Few-Shot Learning 2026-07-26 multimodalvlm
• Prompt Tuning 2026-07-26 finetuningpeft
• Finite Groups and Subgroups 2026-07-26 mathsgroup
• Matrices 2026-07-26
• Replace MathJax with KaTeX in org-mode Exporting 2026-07-25 orgmode
• Closures in Lua 2026-07-24 lua
• Functions in Lua 2026-07-24 lua
• Modules in Lua 2026-07-24
• OCaml Variants and Algebraic Data Types 2026-07-24 ocaml
• Hypothesis Testing 2026-07-24 statistics
• Elisp: Positions 2026-07-23 elisp
• FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction 2026-07-22 speculative_decoding
• MTP: Multi-Token Prediction 2026-07-22 speculative_decoding MTP is a speculative decoding method where the target model includes native multi-token prediction capability. Unlike draft-model-based methods, we don’t need separate draft models.
• PhaseFormer: From Patches to Phases for Efficient And Effective Time Series Forecasting 2026-07-22 time_series
• Elisp Data Structures: Hash Tables 2026-07-22 elisplisp Elisp has 2 types of KV collections. Hash table is unordered set of KV pairs with no duplicate keys and constant access time; Association list is ordered set of KV pairs with possibly repeated keys.
• Elisp Data Structures: Lists 2026-07-22 elisplisp Lisp Lists are implemented as linked lists. Each element can be any type and need not to be of same type.
• Elisp Data Structures: Vectors 2026-07-22 elisplisp Vector implements the array data structures with fixed number of elements. Element’s value can be changed, but the number of elements cannot change.
• Elisp Functions 2026-07-22
• Elisp Symbols 2026-07-22 elisplisp
• Federated Inference: Towards Collaborative and Privacy-Preserving Inference over Edge Devices 2026-07-21 federatedinference
• SensorLM: Learning the Language of Wearable Sensors 2026-07-21 multimodalsensors SensorLM is a family of foundation models that enable wearable sensor data understanding with natural language.
• Surface Integrals 2026-07-21 mathsvector_calculus
• Elisp Arithmetics 2026-07-21 elisplisp
• Elisp Boolean 2026-07-21 elisplisp
• Elisp Conditionals 2026-07-21 elisplisp
• Loops and Iteration in Emacs Lisp 2026-07-21 elisplisp
• Elisp Strings 2026-07-21 elisplisp
• Elisp Variables 2026-07-21 elisplisp
• Print and Output with Elisp 2026-07-21 elisppl We introduce basic methods in Elisp to output and print messages.
• Address Translation: Segmentation 2026-07-20 osmemory The generic technique we’ll be using is called hardware-based address translation.
• Managing Memory in Odin 2026-07-20 odin
• Qwen2.5-VL Technical Report 2026-07-18 multimodalvlm
• Arrays and Slices in Odin 2026-07-18 odin
• Vectors and Array Programming 2026-07-18 odin
• Enums, Unions and Switch 2026-07-17 plodin
• Procedures and Scopes 2026-07-17 odin
• Odin Structs 2026-07-17 plodin
• Unions in Odin 2026-07-17
• defer Statements 2026-07-17 odin
• Maybe Type in Odin 2026-07-17 odin Maybe type is a special union in Odin. Variables can either have no value or some value.
• Clustered Federated Learning 2026-07-16 federatedclustered
• FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments 2026-07-16 federatedclustered
• DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation 2026-07-15 speculative_decoding DSpark is a speculative decoding framework proposed by DeepSeek. Speculative decoding is an acceleration technique for LLM inference.
• Robust Federated Inference 2026-07-15 federatedinference Federated inference is different from federated learning. Federated learning emphasizes the aggregation of models trained on on-edge data; while federated inference focuses on that aggregate the results from distributedly deployed models. This paper focuses on robustness of this paradigm with theoretical formulation and analysis.
• Module Systems in OCaml 2026-07-14 ocamlPLmodule
• Contribution of Buddhism to Chinese Attitude towards Life 2026-07-13 artsbuddhism
• Buddhism and Confucianism - Filial Piety 2026-07-12 artsbuddhism
• Outline of Buddhist Impact on Chinese Culture 2026-07-12 artsbuddhism
• LA-LoRA: Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models 2026-07-09 federatedfinetuninglora This article reviews federated learning under differential privacy federated learning settings.
• Buddhist Schools of Thought and Tradition 2026-07-07 artsbuddhism
• Buddhist and Chinese Attitude to Life 2026-07-07 artsbuddhism We briefly discuss about the similarities and differences between Buddhist and traditional Chinese thoughts, as well as Buddhism’s contribution to Chinese way of life.
• Buddhism in China - Cultural Outline 2026-07-07 buddhism
• Divergence Theorem 2026-07-04 maths
• Green’s Theorem 2026-07-04 maths
• Line Integrals 2026-07-04 mathsvector_calculus
• FM: Generative Models via Flow Matching 2026-07-03 generativeflow_matching Flow matching is an amazingly simple algorithm for generative models. It has been widely adopted in image generation models.
• FlashMask 2026-07-02 attention
• Basic Buddhist Teachings 2026-07-01 artsbuddhism
• Differential Operators 2026-06-30 mathsvector_calculus
• Length of Curves 2026-06-30 mathvector_calculus
• Continuity 2026-06-30 maths
• Differentiation and Derivative 2026-06-30 maths
• C++26 - Static Reflection 2026-06-29 cppplreflection
• SDFT: Self-Distillation Enables Continual Learning 2026-06-28 finetuningdistillation The paper proposes Self-Distillation Fine-Tuning (SDFT) that aims to handle the problem of continual learning: keep learning new skills while not forgetting old skills.
• Local Differential Privacy 2026-06-27 privacy Local differential privacy (LDP) follows a decentralized paradigm that each client perturbs the local data before releasing, preserving populational information while protecting individual privacy. A common statistical tool frequency oracle is built on top of LDP that can handle various data frequency queries.
• TimeXer 2026-06-26 time_series
• Centralized Image Storage with Custom Tag 2026-06-26 techtipsorgemacs I implemented a feature to centrally organize my images under .assets/ folder that supports image preview and exporting to HTML with correct path.
• Diffusion Models 2026-06-24 generativediffusion Diffusion models learn to reverse a gradual noising process, transforming a simple Gaussian prior into samples from a complex data distribution.
• Introduction to Differential Privacy 2026-06-23 privacy Briefly looks at previous approaches to protect privacy and why they fail; then discuss several important concepts in DP and formalize the definition.
• Migrate to jujutsu VCS 2026-06-23 vcsjujutsutechtips jujutsu is a modern version control system that auto-records file edits.
• FedLWS: Federated Learning with Adaptive Layerwise Weight Shrinking 2026-06-22 federated
• Modality Plug-and-Play 2026-06-22 multimodal
• Change of Variable Formula 2026-06-22 mathsprobabilistics
• Latent Variable Models: VAEs 2026-06-21 generativevae VAE extends the idea of variational inference to use neural networks to model both encoder and decoder. For encoder, it uses neural network (e.g. CNN) to study distribution \( q_{\lambda}(\mathbf{z}|\mathbf{x}) \) and outputs a distribution of \( \mathbf{z} \). For decoder, we sample from the distribution \( p(\mathbf{z}) \) and generates images from it through decoder that have learnt \( p(\mathbf{x}|\mathbf{z}) \) from dataset.
• Normalizing Flow Models 2026-06-21 generativenormalizing_flow Autoregressive models provide tractable likelihoods but no direct mechanism for learning features, while VAEs can learn feature representations via latent variables but have intractable marginal likelihoods.
• Variational Inference 2026-06-21 probabilistic Variational inference is the core math principle supporting modern generative models like VAE.
• C++ Type Traits in SFINAE 2026-06-21 cpp SFINAE with type traits is equivalent to C++20 concepts.
• C++ Type Traits 2026-06-21 cpp Type traits define compile-time template-based interfaces to query the properties of types.
• C++20 Concepts 2026-06-21 cppconcepts C++20 concepts provides compile-time checks of template arguments and perform functino dispatch based on properties of types.
• Groups 2026-06-20 mathsgroup
• Generic Algorithm for Integer Programming 2026-06-20 mathsoptimizationinteger_programminggeneric_algorithm
• Overview of Linear Programming 2026-06-20 mathsoptimization Linear Programming is a subset of optimization. This post discusses about general form of linear programming, convexity, and geometry of LP (which is simplex).
• C++26 - Contracts 2026-06-20 cpp Contracts are function modifiers that conducts checks before function execution, similar to Solidity.
• FeDaL: Federated Dataset Learning for General Time Series Foundation Models 2026-06-19 federatedtime_series FeDaL focuses on training time series foundation models (TSFMs) under federated settings. In short, FeDaL decomposes heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowledge; it mitigates local and global biases through bias elimination. FeDaL also shows how data volume, client count and join rate affect model performance under decentralization.
• SageAttention — INT8 Scheme for Inference 2026-06-19 attentionquantization
• Convexity 2026-06-19 mathsoptimization
• Overview of Optimization 2026-06-19 mathsoptimization
• Survey of RL for LRMs 2026-06-18 rlpost_training RL has become an important part for Large Reasoning Models. This article briefly reviews a survey paper on RL for LRMs, focusing on LLMs’ reasoning ability
• Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models 2026-06-18 federatedfinetuning Ferret uses first-order method with shared randomness to enable scalable full-parameter update. It also projects updates to a lower dimensional space to reduce communication overhead, and then reconstructs local updates from lower dimensional space with shared randomness for effective full-parameter global aggregation.
• MapReduce 2026-06-18 csdistributed_system MapReduce used to be a popular paradigm for processing big data.
• How to Read A Paper: A Three-Pass Method 2026-06-18 tips A three-pass method to read a paper.
• Left Recursion 2026-06-17 cscompilerparsing
• Predictive Parsing 2026-06-17 csparsingcompiler Like recursive rescent parsing, but parser can “predict” which production to use next, by looking at the next few tokens (which requires restricted grammars) and no backtracking.
• Terms in RL Post-Training 2026-06-15 rlpost_training Explain terms frequently appearing in AI articles.
• KL Divergence 2026-06-15 mathprobability KL disvergence measures
• Concurrency in Rust 2026-06-14 csplrustconcurrency
• Euler Tour Tree 2026-06-13 acmdata_structuretree Euler tour tree can be used to solve dynamic tree problem. It converts operations into range operations on DFS sequence. Compared to Link-Cut Tree, ETT is more suitable for maintaining subtree information.
• Treap 2026-06-13 acmdata_structuretree Non-rotational treap (FHQ Treap) can be used to maintain sequence operations and balanced tree.
• Literature Review on World Models 2026-06-13 world_model
• Recursive Descent Parsing 2026-06-13 cscompilerparsing A top-down and left-to-right parsing algorithm that terminals are seens in order of appearance in the token stream.
• AST 2026-06-13 cscompiler
• Error Handling in Compilers 2026-06-13 cscompiler
• C++ future 2026-06-13 csplcpp
• C++ Threading 2026-06-13 csplcppthread
• Path to Familiarize with Any PL 2026-06-13 cspl Personal perspective on how to quickly familiarize with any programming language. Also a collection of introductory projects.
• Effective Modern C++14 2026-06-12 csplcpp Effective Modern C++14, Best practices in industry.
• CLIP 2026-06-11 multimodalvlm Pioneering algorithm for image understanding and image-text alignment.
• ViT: Vision Transformer 2026-06-11 vision Vision transformer (ViT) unifies the architecture for vision tasks with transformer, enabling later VLM-related works. The paper URL is An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.
• Array Programming 2026-06-11 csplodinarray
• Types 2026-06-11 csplodin
• Module Management in Rust 2026-06-11 plrust packages, crates, modules, workspaces
• Odin Basic Grammars 2026-06-10 PLodincs
• Cartesian Tree 2026-06-09 acmdata_structuretree
• Lexical Analysis 2026-06-09 cscompilerlexical Lexical analysis is about “classify program substrings according to their role” and then “communicate tokens to the parser”
• Lexical Specification & DFAs 2026-06-09 cscompilerlexical Briefly discuss how to use regex to specify lexemes, also its implementation.
• Parsing: Context-Free Grammars 2026-06-09 csparsingcompiler
• Overview of Internet and Networks: How It Works? 2026-06-09 csnetwork
• Protocol Layers 2026-06-09 networkscs Layering and encapsulation organizes the structure of network.
• Address Space 2026-06-08 osaddressmemory
• Trap Mechanism and Context Switch 2026-06-07 ostrap
• Haskell Basic Grammars 2026-06-07 haskell
• zig Grammars 2026-06-07 zigPL
• DeepSeek V3.2 Technical Report 2026-06-06 attentionrl
• DeepSeek FP8 Training 2026-06-06 trainingquantization Briefly introduces DeepSeek’s FP8 training pipeline, as released in the DeepSeek V3 technical report.
• LoRA 2026-06-06 finetuninglora LoRA has somehow become the standard method for finetuning LLMs.
• Radio - Rate Distortion Optimization for LLM Compression 2026-06-06 compressionquantization Radio provides a novel view for quantization with optimization technique and rate-distortion theory, as well as a closed form for quantization bit. The drawback is that it does not consider hardware efficiency.
• Lichao Tree 2026-06-04 acmdata_structure
• Merging and Splitting Segtree 2026-06-04 acmdata_structuresegtree
• Segtree Basics 2026-06-04 acmdata_structuresegtree
• Split Federated Learning 2026-06-03 federatedsplit Split federated learning is a cross-field of FL and split learning, which can be regarded as a subset of distributed learning with some special settings.
• Android Emulator 2026-06-03 sweandroidemulator Introduces how to use Android Emulator to run android apps for testing
• DeepSeek V3.2 RL Protocol 2026-06-02 rlpost_training
• Automatic Differentiation (AutoGrad) 2026-06-02 trainingautograd
• DeepSeek Sparse Attention 2026-06-02 attentionsparse DeepSeek Sparse Attention (DSA) is introduced along with the DeepSeek V3.2 technical report. It’s designed to reduce computational complexity while preserving model performance in long context scenarios.
• Retrieving Results from Activities 2026-06-02 sweandroid Briefly covers how to get results from activities after closing it, e.g., get file path from file chooser activity.
• Vanilla Attention 2026-06-01 attention
• Principle beyond Apache Spark 2026-06-01 systemsparkdistributed
• Completely Fair Scheduler in Linux 2026-06-01 oslinuxscheduling Linux uses CFS as its process scheduler, apart from textbooks’ MLFQ
• Process Scheduling 2026-06-01 oscpuscheduling
• Limit 2026-06-01 mathanalysislimit
• Run DimensionToTsuLovers on Linux 2026-06-01 linuxwinegame
• FedEx-LoRA: Exact Aggregation for Federated and Efficient Finetuning of Foundation Models 2026-05-31 federatedfinetuninglora Direct LoRA in federated settings causes inaccurate updates. FedEx-LoRA handles this by adding a residual error term to the pretrained frozen weight matrix, achieving exact updates.
• PagedAttention 2026-05-31 inferencekv_cache
• SageAttention v2 — INT4 Scheme 2026-05-31 attentionquantization
• Remote Procedure Call 2026-05-31 distributedrpc
• MultiThreading in Android 2026-05-31 sweandroidmultithreading
• OkHttp Library 2026-05-31 sweandroidnetwork OkHttp is an open-source library for network requests. It’s simple, supports async, filtering, and is extensible, which makes it the standard for modern Android Apps.
• Structural Design Pattern: Adapter 2026-05-31 swedesign_pattern Adapter is a structural design pattern that allows objects with incompatible interfaces to collaborate.
• Design Pattern: Bridge 2026-05-31 swedesign_pattern Bridge is a structural design pattern that lets you split a large class or a set of closely related classes into two separate hierarchies - abstraction and implementation - which can be developed independently of each other.
• Design Pattern: Composite 2026-05-31 swedesign_pattern
• Disk Management and File System in Linux 2026-05-30 linuxdiskfile_system Introduces naming convention of hardware devices.
• Chain of Responsibility 2026-05-30 swedesign_pattern Chain of responsibility is a behavioural design pattern that passes requests along a chain of handlers.
• Abstract Factory 2026-05-30 swedesign_pattern
• Modernity 2026-05-29 modernityphilosophywesternmodern
• Data Storage with Native Android File System 2026-05-29 androidstoragejavakotlin
• Room Persistence Library 2026-05-29 androiddatabase
• Resource Folder in Android Project 2026-05-29 android
• Factory Method 2026-05-29 swedesign_pattern
• SOLID Principles 2026-05-29 swe
• Module Management in Haskell 2026-05-29 haskellmodule
• Kotlin Grammars 2026-05-29 kotlin
• Programming with Apache Spark 2026-05-28 sparkjava
• Programming with Hadoop 2026-05-28 hadooppracticaljava
• Content Providers 2026-05-28 androidstorage
• SQLite in Android 2026-05-28 androidsqlitedatabase
• Data Storage in Android: SharedPreferences 2026-05-28 androiddatabase
• Data Storage in Native Android: Overview 2026-05-28 androiddatabase
• MVVM 2026-05-28 swe
• Triple Products 2026-05-28 mathvector_analysiscalculus
• Dynamic Segtree 2026-05-24 acmdata_structure
• Stokes Theorem 2026-05-24 mathvector_analysiscalculus