(a) Llama? PI

(b) Llama2 NTK

(c) ChunkLlama

- 1.0

6254

6254

6254

st

st

st

18.75 4

18.75 4

18.75 4

-o8

25.04

25.04

25.04

31.25 4

31.25 4

31.25 4

2

3754

2

3754

2

3754

5 43.75 4

5 43.75 4

5 43.75 4

-06

EB 5004

EB 5004

50.04

§ 96.25 4

§ 96.25 4

§ 96.25 4

+04

F625

F625

F625

5 68.754

5 68.754

5 68.754

75.04

75.04

75.04

81.254

81.254

81.254

p02

8754

8754

8754

93.754

93.754

93.754

100.0 4

100.0 4

100.0 4

0.0

4096

6144

8192

10240 12288 14336 16384 18432 20480 24576

4096

6144

8192

10240 12288 14336 16384 18432 20480 24576

4096

6144

8192

10240 12288 14336

16384 18432 20480 24576

‘Token Limit

‘Token Limit

‘Token Limit
