error in R when using a for-loop to extract the list of significant coefficients of autocorrelation - time series

Question

I am trying to run a for-loop in order to create a list/ dataframe object that would give me the set of ACF coefficients that are significantly different to 0 (that are outside of the confidence interval). I am not too sure my for-loop does this, and anyways it gives me the following error message :

 Error in if (acf_res$ci[2, i + 1] < acf_res$acf[i + 1] || acf_res$ci[1,  : 
   missing value where TRUE/FALSE needed

Here is my current code, I am working on a univariate time series of electricity production from 1981 to 1999 included in France :

production_periode_1 <-periode_1$`Production brute d'électricité nucléaire (en GWh)`
prod_periode_1 <- ts(periode_1, frequency=12)
prod_periode_1 <- ts(production_periode_1, start=c(1981,1) , end=c(1999,12), 
frequency=12)
summary(prod_periode_1)
plot.ts(prod_periode_1)
plot(ts.union(prod_periode_1,log(prod_periode_1)))

acf(prod_periode_1,lag.max=150)
acf(diff(prod_periode_1),lag.max=150)
acf(diff(diff(prod_periode_1,12)),lag.max=150)


pacf(prod_periode_1, lag.max = 250)
pacf(diff(prod_periode_1), lag.max = 250)
pacf(diff(diff(prod_periode_1,12), lag.max = 250))


for (i in 1:228) {
  acf_res <- acf(prod_periode_1, lag.max = i, plot = FALSE)
  if (acf_res$ci[2, i+1] < acf_res$acf[i+1] || acf_res$ci[1, i+1] > acf_res$acf[i+1]) {
    print(paste("Coefficient significatif pour lag", i))
  } else {
    print(paste("Pas de coefficient significatif pour lag", i))
  }
}

Here is the structure of my dataset for reproducibility :

 structure(c(36509.514, 34485.002, 33702.518, 31274.906, 30241.116, 
 31542.381), tsp = c(1981, 1981.41666666667, 12), class = "ts")

I would be grateful for any advice !

The value of acf_res$ci is NULL. The acf() function returns an object of type acf. This class does not have a field called "ci". Please see ?acf A confidence interval is shown when you plot an object of type acf. See also: stats.stackexchange.com/questions/211628/… — br00t
– br00t, Commented May 5, 2023 at 13:00
building on @br00ts comment: you can obtain the confidence intervals from acf like so: stackoverflow.com/questions/14266333/… or by using Acf (note capital A) from {forecast}. — I_O
– I_O, Commented May 5, 2023 at 13:10

Martin C. Arnold · Accepted Answer · 2023-05-13 07:49:27Z

2

+25

Here's a proposal using tidyverse which avoids a loop. The idea is to setup a tibble (a data.frame with additional features), compute the CI as done by plot.acf() and check CI exclusion using its columns.

library(tidyverse)

theacf <- acf(prod_periode_1)

tibble(
  sample_ac = theacf$acf[, , 1],
  CI_lower = -qnorm(.975)/sqrt(length(prod_periode_1)),
  CI_upper = -CI_lower,
  significant = !between(sample_ac, CI_lower, CI_upper)
)

For your sample data, this yields

# A tibble: 6 × 4
  sample_ac CI_lower CI_upper significant
      <dbl>    <dbl>    <dbl> <lgl>      
1    1        -0.800    0.800 TRUE       
2    0.495    -0.800    0.800 FALSE      
3    0.0157   -0.800    0.800 FALSE      
4   -0.403    -0.800    0.800 FALSE      
5   -0.426    -0.800    0.800 FALSE      
6   -0.181    -0.800    0.800 FALSE

answered May 13, 2023 at 7:49

Martin C. Arnold

9,7281 gold badge16 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sahil Doshi · Accepted Answer · 2023-05-17 11:49:00Z

The error you're encountering is due to missing values in the confidence interval (acf_res$ci). This happens when the ACF calculation does not provide confidence intervals for certain lags. To address this issue, you can modify your code to skip the missing values when checking for significance. Here's an updated version of your for-loop:

for (i in 1:228) {
  acf_res <- acf(prod_periode_1, lag.max = i, plot = FALSE)
  if (!is.na(acf_res$ci[2, i+1]) && !is.na(acf_res$ci[1, i+1])) {
    if (acf_res$ci[2, i+1] < acf_res$acf[i+1] || acf_res$ci[1, i+1] > acf_res$acf[i+1]) {
      print(paste("Coefficient significatif pour lag", i))
    } else {
      print(paste("Pas de coefficient significatif pour lag", i))
    }
  } else {
    print(paste("Intervalles de confiance manquants pour lag", i))
  }
}

In this updated code, we first check if the confidence interval values at the specified lag (acf_res$ci[2, i+1] and acf_res$ci[1, i+1]) are missing (NA). If they are not missing, we proceed with the significance check as before. If they are missing, we print a message indicating that the confidence intervals are not available for that lag.

This modification should prevent the "missing value where TRUE/FALSE needed" error and provide more informative feedback when confidence intervals are missing.

br00t · Accepted Answer · 2023-05-17 17:07:20Z

0

This code calculates the 95% confidence interval if you assume that the lagged values are white noise:

ci <- 0.95
prod_periode_1 <-  structure(c(36509.514, 34485.002, 33702.518, 31274.906, 30241.116, 
                               31542.381), tsp = c(1981, 1981.41666666667, 12), class = "ts")
summary(prod_periode_1)
plot.ts(prod_periode_1)
plot(ts.union(prod_periode_1,log(prod_periode_1)))

acf(prod_periode_1,lag.max=150)
acf(diff(prod_periode_1),lag.max=150)
# # acf(diff(diff(prod_periode_1,12)),lag.max=150)
# 
pacf(prod_periode_1, lag.max = 250)
pacf(diff(prod_periode_1), lag.max = 250)
# pacf(diff(diff(prod_periode_1,12), lag.max = 250))

for (i in 1:228) {
  message(i)
  lmm <- (length(prod_periode_1) - 1)
  if (i > lmm) {
    message(sprintf('lag.max max value of %s exceeded', lmm))
  } else {
    acf_res <- acf(prod_periode_1, lag.max = i, plot = FALSE)
    clim0 <- qnorm((1 + ci) / 2) / sqrt(acf_res$n.used)
    if ((-clim0 > acf_res$acf[ i + 1 ]) || (clim0 < acf_res$acf[ i + 1 ])) {
      print(sprintf('Coefficient significatif pour lag.max %s', i))
    } else {
      print(sprintf('Pas de coefficient significatif pour lag.max %s', i))
    }  
  }
}

edited May 17, 2023 at 17:07

answered May 10, 2023 at 20:01

br00t

1,6149 silver badges12 bronze badges

2 Comments

gerardlambert Over a year ago

I am not convinced by this code : indeed, when I try to run the code on my series differentiated in seasonality and in trend (replacing prod_periode_1 by diff(diff(prod_periode_1,12)), it indicates that all coefficients are significant, whereas the ACF plot indicates many that are in the confidence interval ie not significantly different to 0. This is because I get the following error message : Error in if (-clim0 < acf_res$acf[i + 1] || clim0 > acf_res$acf[i + 1]) { : missing value where TRUE/FALSE needed

br00t Over a year ago

Apologies, I pasted an incorrect version of the code. The post has been updated.

Collectives™ on Stack Overflow

error in R when using a for-loop to extract the list of significant coefficients of autocorrelation - time series

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related