skills.rest
zjunlp
zjunlpOFFICIAL
Explore all skills by this author →
Website𝕏@ChenHuajunGitHub

deep-neural-feature-ansatz

Train fully connected networks and verify the Deep Neural Feature Ansatz with NTK kernels.

75|7|Updated May 2, 2026
Search Tags:#pytorch#mechanistic interpretability#feature learning#functorch#kernel verification#neural tangent kernel#mlp training
One-click install
npx skills add https://github.com/zjunlp/Mechanist --skill deep-neural-feature-ansatz-zjunlp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: deep-neural-feature-ansatz
Source: https://github.com/zjunlp/Mechanist/tree/main/skills/mechanism-skills/neural-feature-learning/eigenvector-feature-direction
Command: npx skills add https://github.com/zjunlp/Mechanist --skill deep-neural-feature-ansatz-zjunlp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, torchvision, functorch, numpy, and includes scripts (resource) components.

What problem does it solve? Verifying whether trained neural networks learn features beyond the kernel regime requires training networks, saving initialization checkpoints, and computing layer-wise NTK/NNGP kernel matrices, which is tedious to set up from scratch. ## Core Features & Use Cases - Network Training: Train configurable fully connected networks (depth, width, learning rate, SGD/Adam) on CIFAR-10, CIFAR-100, SVHN, MNIST, and FashionMNIST, saving both trained and initialization checkpoints. - Ansatz Verification: Load saved networks and compute layer-wise feature kernels, NTK, and NNGP matrices via functorch to compare trained representations against their initialization. - Use Case: Reproduce experiments from the paper "The Deep Neural Feature Ansatz" (arXiv:2212.13881) by training a 4-layer, 512-wide MLP on CIFAR-10 and checking whether learned features match kernel-regime predictions. ## Quick Start Train a fully connected network on CIFAR-10 and verify the Deep Neural Feature Ansatz by comparing its layer-wise kernels against the initialization checkpoint.

Frequently Asked Questions about deep-neural-feature-ansatz

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify the Deep Neural Feature Ansatz on a trained network?▼

Train a network with train_network.py, which saves both the trained model and its initialization to saved_nns/. Then run verify_ansatz.py to load both checkpoints, compute layer-wise feature kernels, and compare trained versus initialization representations.

How to compute NTK and NNGP kernels in PyTorch?▼

Use functorch's vmap with jacrev or grad to compute per-sample Jacobians over data batches, then form kernel matrices without explicit loops. This requires PyTorch 1.13 with functorch installed via pip.

What datasets does the DNFA codebase support?▼

The dataset utilities support CIFAR-10, CIFAR-100, SVHN, MNIST, and FashionMNIST. They include one-hot encoding, train/validation splitting by proportion, and subsampling to fixed train and test counts.

Does functorch work with newer PyTorch versions?▼

The skill targets PyTorch 1.13 with functorch installed separately via pip. Newer PyTorch versions integrate functorch as torch.func but may have API differences, so the provided conda environment is recommended.

Why does kernel computation run out of memory?▼

Kernel matrix computation scales quadratically with sample count and is memory-intensive on large networks. Use small subsets of 500 to 1000 samples for kernel evaluation to avoid OOM errors.

Related Skills

Explore More Education & Research Skills→
lazyFrogLOLlazyFrogLOL
127

caffe-cifar-10

Compile Caffe from source and train it on CIFAR-10.

Community
Advanced
ShiienShiien
7

experiment-analyze

Parse experiment metrics and logs to assess significance against baselines.

Community
Intermediate
Tnemo65Tnemo65

experiment-tracking-swanlab

Track ML experiments by logging runs, metrics, and configurations locally with SwanLab.

Community
Intermediate
thistleknotthistleknot

generalization-theory

Diagnose memorization versus signal learning using empirical neural tangent kernel partitioning.

Community
Advanced