JS 的 AI 時(shí)代來(lái)了！
JS-Torch 簡(jiǎn)介
JS-Torch[1] 是一個(gè)從零開(kāi)始構(gòu)建的深度學(xué)習(xí) JavaScript 庫(kù)，其語(yǔ)法與 PyTorch[2] 非常接近。它包含一個(gè)功能齊全的張量對(duì)象（可跟蹤梯度）、深度學(xué)習(xí)層和函數(shù)，以及一個(gè)自動(dòng)微分引擎。
圖片
PyTorch 是一個(gè)開(kāi)源的深度學(xué)習(xí)框架，由 Meta 的人工智能研究團(tuán)隊(duì)開(kāi)發(fā)和維護(hù)。它提供了豐富的工具和庫(kù)，用于構(gòu)建和訓(xùn)練神經(jīng)網(wǎng)絡(luò)模型。PyTorch 的設(shè)計(jì)理念是簡(jiǎn)單、靈活，以及易于使用，它的動(dòng)態(tài)計(jì)算圖特性使得模型的構(gòu)建更加直觀和靈活。
你可以通過(guò) npm 或 pnpm 來(lái)安裝 js-pytorch:
npm install js-pytorch
pnpm add js-pytorch或者在線體驗(yàn) js-pytorch 提供的 Demo[3]：
圖片
https://eduardoleao052.github.io/js-torch/assets/demo/demo.html
JS-Torch 已支持的功能
目前 JS-Torch 已經(jīng)支持 Add、Subtract、Multiply、Divide 等張量操作，同時(shí)也支持Linear、MultiHeadSelfAttention、ReLU 和 LayerNorm 等常用的深度學(xué)習(xí)層。
Tensor Operations
- Add
- Subtract
- Multiply
- Divide
- Matrix Multiply
- Power
- Square Root
- Exponentiate
- Log
- Sum
- Mean
- Variance
- Transpose
- At
- MaskedFill
- Reshape
Deep Learning Layers
- nn.Linear
- nn.MultiHeadSelfAttention
- nn.FullyConnected
- nn.Block
- nn.Embedding
- nn.PositionalEmbedding
- nn.ReLU
- nn.Softmax
- nn.Dropout
- nn.LayerNorm
- nn.CrossEntropyLoss
JS-Torch 使用示例
Simple Autograd
import { torch } from "js-pytorch";
// Instantiate Tensors:
let x = torch.randn([8, 4, 5]);
let w = torch.randn([8, 5, 4], (requires_grad = true));
let b = torch.tensor([0.2, 0.5, 0.1, 0.0], (requires_grad = true));
// Make calculations:
let out = torch.matmul(x, w);
out = torch.add(out, b);
// Compute gradients on whole graph:
out.backward();
// Get gradients from specific Tensors:
console.log(w.grad);
console.log(b.grad);Complex Autograd (Transformer)
import { torch } from "js-pytorch";
const nn = torch.nn;
class Transformer extends nn.Module {
constructor(vocab_size, hidden_size, n_timesteps, n_heads, p) {
super();
// Instantiate Transformer's Layers:
this.embed = new nn.Embedding(vocab_size, hidden_size);
this.pos_embed = new nn.PositionalEmbedding(n_timesteps, hidden_size);
this.b1 = new nn.Block(
hidden_size,
hidden_size,
n_heads,
n_timesteps,
(dropout_p = p)
);
this.b2 = new nn.Block(
hidden_size,
hidden_size,
n_heads,
n_timesteps,
(dropout_p = p)
);
this.ln = new nn.LayerNorm(hidden_size);
this.linear = new nn.Linear(hidden_size, vocab_size);
}
forward(x) {
let z;
z = torch.add(this.embed.forward(x), this.pos_embed.forward(x));
z = this.b1.forward(z);
z = this.b2.forward(z);
z = this.ln.forward(z);
z = this.linear.forward(z);
return z;
}
}
// Instantiate your custom nn.Module:
const model = new Transformer(
vocab_size,
hidden_size,
n_timesteps,
n_heads,
dropout_p
);
// Define loss function and optimizer:
const loss_func = new nn.CrossEntropyLoss();
const optimizer = new optim.Adam(model.parameters(), (lr = 5e-3), (reg = 0));
// Instantiate sample input and output:
let x = torch.randint(0, vocab_size, [batch_size, n_timesteps, 1]);
let y = torch.randint(0, vocab_size, [batch_size, n_timesteps]);
let loss;
// Training Loop:
for (let i = 0; i < 40; i++) {
// Forward pass through the Transformer:
let z = model.forward(x);
// Get loss:
loss = loss_func.forward(z, y);
// Backpropagate the loss using torch.tensor's backward() method:
loss.backward();
// Update the weights:
optimizer.step();
// Reset the gradients to zero after each training step:
optimizer.zero_grad();
}有了 JS-Torch 之后，在 Node.js、Deno 等 JS Runtime 上跑 AI 應(yīng)用的日子越來(lái)越近了。當(dāng)然，JS-Torch 要推廣起來(lái)，它還需要解決一個(gè)很重要的問(wèn)題，即 GPU 加速。目前已有相關(guān)的討論，如果你感興趣的話，可以進(jìn)一步閱讀相關(guān)內(nèi)容：GPU Support[4] 。
參考資料
[1]JS-Torch: https://github.com/eduardoleao052/js-torch
[2]PyTorch: https://pytorch.org/
[3]Demo: https://eduardoleao052.github.io/js-torch/assets/demo/demo.html
[4]GPU Support: https://github.com/eduardoleao052/js-torch/issues/1
