beam-search classification MLP
beam-search based DeepSpeed implementation for learning inference.
- Input
- 966-dim embedding
- Encoder
- 48 x MLP with 24 heads
- Output
- perplexity projection
Training config
optimizer=SGD, lr=0.583, scheduler=cyclic, warmup=1342