Title
Asymptotic Behavior of Sequence Models
Abstract
In this paper we study the limiting dynamics of a sequential process that generalizes Pólya’s urn. This process has been studied also in the context of language generation, discrete choice, repeat consumption, and models for the web graph. The process we study generates future items by copying from past items. It is parameterized by a sequence of weights describing how much to prefer copying from recent versus more distant locations. We show that, if the weight sequence follows a power law with exponent α ∈ [0, 1), then the sequences generated by the model tend toward a limiting behavior in which the eventual frequency of each token in the alphabet attains a limit. Moreover, in the case α > 2, we show that the sequence converges to a token being chosen infinitely often, and each other token being chosen only constantly many times.
Year
DOI
Venue
2020
10.1145/3366423.3380044
WWW '20: The Web Conference 2020 Taipei Taiwan April, 2020
Keywords
DocType
ISBN
urn processes, power laws, copying models
Conference
978-1-4503-7023-3
Citations 
PageRank 
References 
0
0.34
0
Authors
3
Name
Order
Citations
PageRank
Flavio Chierichetti162639.42
Ravi Kumar2139321642.48
Andrew Tomkins393881401.23