(a) Intra-chunk Attention

(b) Inter-chunk Attention

(c) Successive-Chunk Attention

pintra

Chunk1

Chunk2

Chunk1

Chunk2

pinter

P Intra

Chunk1. ———_,

Chunk2. ——__, psucc

Qa

Qa

ae

Qa

‘(afey | | |

Lj} tty te tT tT

qi

SOEEEEEEEEEE

Li tty te tT tT | |e

pf | | | TT

qt

f

Li tt} tt} tT tT TT

8

‘fale || | | |

Li | | tt} te tT TT |

Soc Sr.

s[5 [4] 3| | | | | |

> &

Fs {s[4|a

0

ofel7fefsi4] | | | | | |»

of6|s5|4]3 mcs

=

=

LT tT | aye |

ofel7fefs|4] | | | | | |»

+[7]6| 5] 4| | | | |?

ofel7{s|si4] | | | | | ie

2[6|7|6| 5 |

| | ls

sf | | | tt fe a

ofel7je|s}4] | | | | | ie

3{9|8|7]6|

|| ie

st | | | | fats a

ofel7je|s}4] | | | | | |e

¢{ole|7|6{5[4] 4) 9

{| | | | | [stats

[ole|7felstay | | | | | |»

sfo|s|7] 6.

[o[e[7e[s [a] >| + [3 [2 iaaiay »

5] 4

2

5

4

5 @ 1

3 4

5

2 5 4 5 6 1

2

3 4 =5

PR @ 1

2 3 4 5 @ 1

2 3 4

Ww=4

PR @ 1

Pres

Position ide of k

Peo 1

P,$ Posttton ids of k

Pr

Position ids of k
