Taylor's Law in Innovation Processes

Abstract

Taylor's law quantifies the scaling properties of the fluctuations of the number of innovations occurring in open systems. Urn-based modeling schemes have already proven to be effective in modeling this complex behaviour. Here, we present analytical estimations of Taylor's law exponents in such models, by leveraging on their representation in terms of triangular urn models. We also highlight the correspondence of these models with Poisson-Dirichlet processes and demonstrate how a non-trivial Taylor's law exponent is a kind of universal feature in systems related to human activities. We base this result on the analysis of four collections of data generated by human activity: (i) written language (from a Gutenberg corpus); (ii) an online music website (Last.fm); (iii) Twitter hashtags; (iv) an online collaborative tagging system (Del.icio.us). While Taylor's law observed in the last two datasets agrees with the plain model predictions, we need to introduce a generalization to fully characterize the behaviour of the first two datasets, where temporal correlations are possibly more relevant. We suggest that Taylor's law is a fundamental complement to Zipf's and Heaps' laws in unveiling the complex dynamical processes underlying the evolution of systems featuring innovation.

Keywords: Poisson–Dirichlet process; Pólya’s urn; Taylor’s law; adjacent possible; innovation dynamics; triangular urn schemes.

Figure 1

Taylor’s law in the urn model with triggering.Left: Taylor’s law from 100 realizations of the stochastic process described in Section 2 (the urn model with triggering), for each of the indicated values parameters. The values of parameters are chosen in order to have a representative curve for each of the analyzed regimes, i.e., $\rho <\nu /2$, $\rho =\nu /2$, $\nu /2<\rho <\nu$, $\rho =\nu$, $\rho >\nu$. Each realization is a sequence of ${10}^6$ elements. Right: Taylor’s law from the same sequences as in the left side picture, individually reshuffled so that to loose the temporal order (refer to the parallel file random reshuffling procedure discussed in Section 5 and in Figure 2).

Figure 2

Shuffling procedures. In this example we consider three different streams A, B, C, consisting of five tokens each. When the analysis is carried in parallel the streams are aligned respecting their natural order (left panel). In the parallel file random case (middle panel), each stream is reshuffled singularly. Eventually, the parallel random case shuffles the elements all together (right panel).

Figure 3

Taylor’s law in real systems and in their randomized instances. The standard deviation $\sigma \left(N\right)$ of the number of different tokens after N total tokens appeared, is plotted vs the average number of different tokens $\mu \left(N\right)$ in four different datasets. The shuffled counterparts are also evaluated. The shuffling schemes are shown in Figure 2.

Figure 4

Stability of Taylor’s law results in the Gutenberg corpus.Left: the analogous of Figure 3 (top left) for three different sets of $M=100$ books from the Gutenberg corpus. Right: as in Figure 3 (top left), with 20 different realizations of the parallel file random reshuffling procedure. We see that the difference between the curve referred to the ordered sequences and those referred to the reshuffled ones is much higher than fluctuations due to different realizations of the reshuffling.

Figure 5

Taylor’s law in the urn model with triggering and quenched stochasticity of the parameters and in the urn model with semantic triggering.Top: Taylor’s law in the urn model with triggering, with parameter’s $N_0=100$, $\rho =1$ and $\nu$ is randomly extracted for each simulation of the process from a uniform distribution on the interval $(0,1)$ (left) and from an exponential distribution on the interval $(0,1)$ and parameter $\lambda =1$ (right), as discussed in the main text. Center: Taylor’s law in the urn model with triggering, with parameter’s respectively: (left) $N_0=100$, $\nu =2$, $\rho =3+r_i$, with $r_i$ randomly extracted for each simulation of the process from an exponential distribution with mean $\overline{r_i}=1$; $\nu =2$, $\rho =3$, $N_0=1+n_i$, with $n_i$ randomly extracted for each simulation of the process from an exponential distribution with mean $\overline{n_i}={10}^4$. Bottom: (left) Taylor’s law in the urn model with semantic triggering, with parameters $N_0=100$, $\nu =6$, $\rho =9$, $\eta =0.6$; (right) Taylor’s law in the urn model with semantic triggering, with parameters $\nu =2$, $\rho =3$, $\eta =0.6$, $N_0=1+n_i$, with $n_i$ randomly extracted for each simulation of the process from an exponential distribution with mean $\overline{n_i}={10}^4$. The parameters of the simulations were chosen such to lie in the regime $\rho <\nu$. The parameter $\eta =0.6$ used in the bottom graphs was chosen in the regime where the Heaps’ and Zipfs’ laws feature exponents compatible with those observed in real systems. In all the figures the curves for the Taylor’s law are constructed from 100 independent realizations of the process (M = 100 in Equation (16)).