SciELO - Scientific Electronic Library Online

 
vol.11 número3Escala de Crenças Irracionais aplicadas a atletas: Proposta de interpretação através do Modelo de Créditos ParciaisAssociação e poder discriminatório da atividade física para a prevenção da sarcopenia em mulheres pós-menopáusicas índice de autoresíndice de assuntosPesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

  • Não possue artigos similaresSimilares em SciELO

Compartilhar


Motricidade

versão impressa ISSN 1646-107X

Motri. vol.11 no.3 Ribeira de Pena set. 2015

https://doi.org/10.6063/motricidade.4100 

ORIGINAL ARTICLE

 

The agreement between physical activity time reported by the IPAQ and accelerometer in postmenopausal women

 

Concordância entre o tempo de atividade física avaliado pelo IPAQ e acelerômetro em mulheres pós-menopausa.

 

 

Clara Suemi Costa Rosa1; Fabrício Eduardo Rossi1,*; Camila Buonani2; Rômulo Araújo Fernandes2; Henrique Luiz Monteiro3; Ismael Forte Freitas Junior2

1 Universidade Estadual Paulista, Instituto de Biociências, Rio Claro – SP – Brazil
2 Universidade Estadual Paulista, Faculdade de Ciências e Tecnologia, Presidente Prudente – SP – Brazil
3 Universidade Estadual Paulista, Faculdade de Ciências, Bauru – SP – Brazil

 

 


ABSTRACT

The objective of this study was to assess, in a sample composed by postmenopausal women, the agreement between physical activity time reported by short version of the IPAQ and measured by accelerometer in different intensities. The participants consisted of postmenopausal women aged ≥ 50 years (N = 60, mean age 61.5±7.5 years). They completed the IPAQ (short version) and then wore an accelerometer for one week. Statistical analysis was composed by the Kappa index, McNemar’s test, specificity and sensitivity, Spearman’s rank correlation and the Wilcoxon test. The amount of time spent in moderate and vigorous physical activity had weak correlation (r=-0.04-0.28) between accelerometer and IPAQ. IPAQ overestimated the minutes spent in moderate and vigorous intensities and underestimated the time spent in light activities. There was not significant agreement between instruments (kappa index= 0.151±0.111; p= 0.185; sensibility, 46% and specificity, 28%). There was a significant inconsistence for the time of physical activity reported by IPAQ.

Keywords: aging, menopausal, accelerometer, questionnaire, agreement


RESUMO

O objetivo deste estudo foi avaliar a concordância entre o tempo de atividade física relatada pela versão curta do IPAQ e medida por acelerômetro em diferentes intensidades. As participantes foram mulheres na menopausa com idade ≥ 50 anos (N = 60, idade média de 61,5 ± 7,5 anos). Elas completaram o IPAQ (versão curta) e depois usaram um acelerômetro por uma semana. A análise estatística foi composta pelo índice de Kappa, teste de McNemar, sensibilidade e especificidade, correlação de Spearman e o teste de Wilcoxon. O tempo gasto em atividade física intensa apresentou fraca relação (r=-0,04 – 0,28) entre acelerômetro e IPAQ. IPAQ superestimou os minutos gastos em intensidades moderada e vigorosa e subestimou o tempo gasto em atividades leves. Não houve concordância significativa entre os instrumentos utilizados para medida do nível de atividade física (índice kappa = 0,151 ± 0,111, p = 0,185; sensibilidade, 46% e especificidade, 28%). Houve uma inconsistência significativa para o tempo de atividade física relatada pelo IPAQ.

Palavras-chave: envelhecimento; menopausa; acelerômetro, questionário.


 

 

INTRODUCTION

Daily physical activity (PA) has been pointed out as important behavior in relation to the prevention of diseases and mortality (Ahmed, Blaha, Nasir, Rivera, & Blumenthal, 2012), mainly in older adults (Centers for Disease Control and Prevention (CDC), 2005). The amount of fat and fat-free mass in adults change with increasing age as a function of a variety of factors, from PA to menopausal, and has been mentioned as a major cause of disease in postmenopausal women (Poehlman, Toth, & Gardner, 1995).

Menopause not only has an impact on women’s reproductive system, but is also responsible for significant changes in body composition, with decreasing of fat-free mass around 3.0 kg, 2.5 kg increase in fat mass and a decline of up to 100 kilocalories per day in resting metabolic rate when compared to premenopausal women (Poehlman et al., 1995). Furthermore, the prevalence of physical inactivity reaches 30% in postmenopausal women (CDC, 2005), factors that may lead to reduced muscle mass and strength, impair locomotion and balance (Rebelatto & Castro, 2007), may increase the number of falls and successively decrease the quality of life causing dependence to perform activities of daily living.

Regarding PA monitoring, there are many technics to measure it, which are usually divided into laboratorial and field methods (Siervo & Jebb, 2010). Laboratory methods include greater objectivity and precision, but require sophisticated and expensive equipment and, from an epidemiological point of view, these methods are not applicable. On the other hand, motion sensors, such as accelerometers, estimate the energy expenditure and intensity of PA (Chen & Bassett, 2005) and have been used in epidemiological settings (Tudor-Locke, Johnson, & Katzmarzyk, 2010). As a field method, questionnaires are less complex, have lower cost and represent simple instruments for assessment of PA. However, questionnaires have some limitations based on recall bias a poor self-perception of PA intensity, particularly in older adults that may have difficulty with memory and cognition; in addition changes in lifestyles such as engagement in less structured high level of PA may compromise the survey accuracy (Grimm, Swartz, Hart, Miller, & Strath, 2012).

During the last decade the International Physical Activity Questionnaire (Craig et al., 2003) has received special attention by the scientific literature and has been widely used worldwide. The IPAQ was created to fulfill the necessity to standardize the assessment of PA in different cultures around the world (Craig et al., 2003), consequently searching for its adaptation the questionnaire has suffered wide cultural modification and changes to produce more precise information. Highlighting this, in a recent publication Hallal et al. (2010) called attention for main lessons learned from this series of adaptations and emphasizing the requirement to test an instrument previously, even if they have been previously using similar populations.

In a brief search in PubMed, where the keyword “IPAQ” was used, more than 280 articles were found. Of these articles approximately 75 reported validity and reliability procedures, however, there was no data reporting the agreement between PA time reported by the IPAQ questionnaire and the accelerometer for postmenopausal women.

Additionally, the validity of IPAQ, using a motion detector, reported in the 12-Country Reliability and Validity Study of IPAQ short-version is modest r=0.30. However correlation found for the Brazilian sample (n=28, age: 42.9±14.2) was low and negative (r=-0.12) (Craig et al., 2003).

Therefore, the objective of this study was to assess, in a sample composed of postmenopausal women, the agreement between PA time reported by the short version of the IPAQ and measured by an accelerometer for different intensities.

 

METHOD

Participants

The sample consisted of postmenopausal women aged ≥ 50 years, who were selected to engage in an exercise program promoted by the Department of Physical Education of the Universidade Estadual Paulista (UNESP), located in the city of Presidente Prudente, Brazil. The dissemination of the study was done through newspapers, television and the Internet. Women interested in participating contacted the researchers responsible. To participate in this study the inclusion criteria were: 1) female, 2) postmenopausal (no menstruation for 1 year or more), 3) aged 50 to 79 years on the date of the assessment, 4) no regular exercise for a period of 6 months prior to the valuation date (they were asked about exercising in their leisure-time, such as, sports). The present study reports only the baseline measures of the exercise program. Considering an a-error of 5%, b-error of 80% and a minimum correlation of 0.38 (Oyeyemi et al., 2014), the sample size considered for this study was 53 women, then expecting losses our study select 67 women to participate. However, given some limitations in the questionnaire and accelerometer data, seven subjects were excluded from this study. The procedures used in this study meet the criteria of the Ethics in Human Research according to resolution number 196/96 of the Brazilian Health Ministry and the study was previously approved by the Ethical Research Committee of the UNESP (number 64/2011). All individuals signed a written consent form.

Materials and Procedures

The study has a cross-sectional design and compared the time of PA reported by a questionnaire with that measured by an accelerometer. The participants were given detailed instructions on wearing an accelerometer for a consecutive 8-day period. The questionnaire was applied through a face-to-face interview by a trained researcher, fallowing the instruction from the Guidelines for Data Processing and Analysis of the International Physical Activity Questionnaire (2005).

The International Physical Activity Questionnaire (IPAQ) was applied in its reduced version (Craig et al., 2003) for assess the usual week of PA. The International Group proposed the IPAQ for Consensus on Measures of Physical Activity, under the seal of the World Health Organization, with representatives from 25 countries, including Brazil. It is an instrument developed in order to estimate the level of PA practice in populations of different countries and sociocultural contexts (Craig et al., 2003).

The IPAQ is available in different languages, including Portuguese, and therefore required no translation. There are two versions of the IPAQ, a long and a short form. We chose to analyze the questionnaire in its short form, as it is acknowledged to be the more practical version. This version is composed of eight questions and information to estimate the time spent per week on different intensities of PA (moderate, vigorous, walking and sedentary behaviors, such as sitting). In this study, we refer to IPAQ walking intensity as light PA.

PA was measured by accelerometers (ActiGraph GT3X, Pensacola, FL). The devices are lightweight (27 g) and small (size 3.8 cm x 3.7 cm x 1.8 cm), and are designed to record movements in the three orthogonal planes: anteroposterior, mediolateral, and vertical. The accelerometer (ActiGraphGT3X) measures and records changes in acceleration whose magnitudes cover approximately 0.05-2.5 g (g = 9.8 m/s2) within a frequency range of 0.25- 2.5 Hertz. The output of the accelerometer is digitized by an Analogue-Digital converter 12 bit at a rate of 30 times per second (30 Hz). The PA intensity was measured in 60-s epochs, this epoch is related to low intensity and long duration patterns of PA (Trost, McIver, & Pate, 2005), which is standard for monitoring free-living adults (Chen & Bassett, 2005).

Accelerometers were placed on the individuals’ waists using an elastic band. The participants used the equipment for eight days (only seven were full days). Instructions for use were given previously. The accelerometer had to be used all day during the hours that the individual remained awake and should only be removed when there was contact with water (personal hygiene or water activities). Specific software (The ActiLife5 Data Analysis Software by Actigraph) was used to analyze the data. We included only full days of monitoring in the database. Zero hour consecutive counts were considered to be periods that the individual was not wearing the accelerometer, days containing less than ten hours of monitoring were excluded from the analysis, as they had the power to increase variability (Craig et al., 2003). At the end, each participant had at least five full days of monitoring (Trost et al., 2005).

The PA intensities were determined as follow: time spent in time spent in light-intensity (100-1951 counts/min), moderate-intensity (1952–5724 counts/min) and vigorous-intensity (≥ 5,725 counts/min) (Freedson, Melanson, & Sirard, 1998). Sedentary time cut-point for accelerometer was not computed because our analysis did not distinguish between sleeping and waked time, differently from IPAQ, which computed only sedentary time from waked time. In the present study counts per minute in the vertical vector was used according to previous cut points available in the literature, since, nowadays, there are no three vector’s thresholds to distinguish sedentary time from light PA. For both instruments the total time spent on moderate and vigorous activity was computed and then participants were classified as: sufficiently active (≥150 min per week) and insufficiently active (<150 min per week), according to the cutoff point proposed by the American College of Sports Medicine (Haskell et al., 2007).

Data Analysis

For analysis of the information we utilized non-parametric statistics, since the data presented non-parametric distribution (Kolmogorov-Smirnov test). Descriptive statistics were composed of values of median and interquartile range. For numerical comparisons between the questionnaire and accelerometer the Wilcoxon test was applied with effect size (r) calculated. Spearman’s rank-order correlations were used to compare time spent in different PA intensities between the IPAQ and accelerometer. For categorical data, the McNemar’s test compared the proportion. To analyze the agreement between IPAQ and accelerometer to identify subjects that achieve the PA recommendations (≥150min/week), Kappa index was tested. The ROC curve and its parameters (sensibility, specificity, area under curve – AUC) were calculated to classify individuals according to the global standard guideline for sufficient PA (≥150min/week). The alpha level was set at 0.05 and all statistical analysis was performed using the software BioEstat (release 5.0).

 

RESULTS

There were 60 women with age ranging from 50 to 79 years old (mean age 61.5±7.5 years). The median body mass index was 29.1(8.1) kg.m-2. Table 1 shows the median and interquartile range for both instruments. The amount of time spent in different PA intensities was compared according to the PA instruments (Table 1). IPAQ overestimated the minutes spent on moderate and vigorous intensities (small [r=0.30] to medium [r=0.60] effect size). Conversely, the time spent on light activities were underestimated (medium effect size = 0.45).

The correlation between the IPAQ and accelerometer was identified by Spearman rank (Figure 1). According to the accelerometer and IPAQ, the amount of time spent in PA intensities had a weak relationship (ranging from 0.13 to 0.28). However, light PA only had a significant correlation (r= 0.279; p= 0.031). On the other hand, the relationship for vigorous activity presented a negative pattern (r= -0.041; p=0.756). All correlation consequently, showed small coefficient of determination.

 

 

The percentage of subjects classified as physically active was 40% and 65% for the accelerometer and IPAQ (McNemar test with p= 0.006 [Figure 1]), respectively. The ROC curve analysis indicated an AUC of 0.58 (IC 95% 0.43 - 0.73, p=0.026). The sensitivity of the IPAQ to find insufficiently active individuals was 46%, whereas, 28% (the specificity) of those meeting the international recommended guidelines as determined by the accelerometer were identified by the IPAQ. There was not a significant agreement between the two PA instruments (kappa index= 0.151±0.111; p=0.185).

 

 

 

DISCUSSION

In the presented cross-sectional study data reported the agreement between IPAQ Short version and accelerometer in postmenopausal women. Our results identified that there were weak agreement between PA time assessed by the questionnaire and accelerometer (r=0.13 to 0.29). Although these results are similar to what was reported in the 12-Country Reliability and Validity Study of IPAQ short-version (r=0.30), the IPAQ significantly underestimated the time spent light activities and overestimated the time spent on moderate and vigorous PA when the accumulated accelerometer minutes per day were examined. Similar our results, Johnson-Kozlow, Sallis, Gilpin, Rock, and Pierce (2006) studying women who had been diagnosed with breast cancer (average age 57 years) reported that the maximum total PA score for the IPAQ was approximately three to four times greater than that obtained on the accelerometer. The IPAQ significantly overestimated moderate PA by about 225 minutes per week or 239% (p < 0.001) and significantly overestimated vigorous PA by about 31 minutes per week or 342% (p < 0.001). Similarly, Grimm, Swartz, Hart, Miller, and Strath (2012) identified that women presented 27.3±20.0 min/week for the accelerometer versus 49.5±47.6 min/week for the IPAQ for moderate intensity and 2.0±6.9 min/week for the accelerometer versus 37.2 ± 50.7 min/week for the IPAQ for vigorous PA (long-form questionnaire).

The identification of correlates of PA is important to target the elaboration of more effective public health programs related to PA promotion. Therefore, worldwide there have been a wide variety of epidemiological studies, in which the main purpose has been to determinate some correlates of PA. From a statistical point of view, this bias observed for PA provided by IPAQ can affect the relationship observed between PA and its determinants and, therefore, some determinants should not be detected (false negative), in addition to, the opposite situation, which can also be observed (false positive).

The above-mentioned overestimation also affected the prevalence of physically active women, in which the IPAQ generated a rate 62.5% higher than the accelerometer. During the last decades, developing nations (such as Brazil) have observed an increase in chronic diseases (Cesse, Carvalho, Souza, & Luna, 2009). This worrying trend has been associated with a decrease in PA practice (Fernandes et al., 2011; Fernandes & Zanesco, 2010). On the other hand, surprisingly, studies using IPAQ have identified an increase in PA practice in these nations (Matsudo et al., 2010). The elaboration of a linear regression model to describe/adjust this overestimation could be indicated; however, the absence of normal distribution among PA data and the low relationship between the IPAQ and accelerometer measures make this elaboration a hard task. Our findings indicate that the IPAQ overestimation of PA practice should be taken into account in prevalence studies and hence consistent efforts are necessary to correct this limitation.

There are many hypotheses to explain the lack of disagreement between self-reported and objective instrument in this population. Among possibilities, such as, lack of comprehension due to cognitive capacity and educational level (not verified in this study), and memory, it is possible, also, that the overestimation of time in PA remain due to the PA held during domestic activities, since we have a sample of women with an average age of 61 years. Unlike leisure activities, those performed at work or domestic environment have considerable variation from day to day and even on the same day in terms of the kind of intensity, duration and rest periods, causing inconsistence in the self-reported PA (Hallal et al., 2010).

The study has limitations. The IPAQ assesses PA in different domains for a variety of activities; however, many of these activities may be under-estimated by accelerometers. The accelerometer is known not to measure all PA as it underestimates the one conducted primarily above the waist, such as sweeping or carrying a load, or activity done on an incline, such as running up a hill, and does not measure water-based activity at all (Chen & Bassett, 2005). Yet, our study used uniaxial vector (vertical vector) that can underestimate patterns of PA compared with cut points for triaxial accelerometers. Nevertheless, the accelerometer counts used to assess PA behavior in this study may not hold true for this population, as well as the cut point utilized (Ward, Evenson, Vaughn, Rodgers, & Troiano, 2005). In addition to these limitations, our participant sample included a range of normal, overweight, and obese individuals, and to our knowledge no known published work reports on the comparability of the IPAQ and ACC across BMI classification. Furthermore, the wide range of age that composed the sample could play an important role in the analysis since older people may have different patterns of PA. Future studies are warranted in this area to further examine and verify BMI and age.

 

CONCLUSION

In summary, the presented study identified low agreement between PA time provided by the IPAQ (short version) and accelerometer. There was a significant inconsistency between the time of PA reported by the IPAQ (short version), which overestimated moderate and vigorous activities and underestimated the light one.

These study aim to guide future studies as to the best methods for assessing PA in post-menopausal women, not only to provide appropriate surveillance but also to facilitate comparisons with worldwide populations. We recommend caution when using IPAQ-short version in this population, mainly because of misreporting time of PA.

 

REFERENCES

Ahmed, H. M., Blaha, M. J., Nasir, K., Rivera, J. J., & Blumenthal, R. S. (2012). Effects of physical activity on cardiovascular disease. The American Journal of Cardiology, 109(2), 288–295. http://doi.org/10.1016/j.amjcard.2011.08.042        [ Links ]

Centers for Disease Control and Prevention (CDC). (2005). Trends in leisure-time physical inactivity by age, sex, and race/ethnicity--United States, 1994-2004. Morbidity and Mortality Weekly Report, 54(39), 991–994.         [ Links ]

Cesse, E. A. P., Carvalho, E. F. de, Souza, W. V. de, & Luna, C. F. (2009). Mortality trends due to circulatory system diseases in Brazil: 1950 to 2000. Arquivos Brasileiros De Cardiologia, 93(5), 490–497. http://doi.org/10.1590/S0066-782X2009001100009        [ Links ]

Chen, K. Y., & Bassett, D. R. (2005). The technology of accelerometry-based activity monitors: current and future. Medicine and Science in Sports and Exercise, 37(11 Suppl), S490–500.         [ Links ]

Craig, C. L., Marshall, A. L., Sjöström, M., Bauman, A. E., Booth, M. L., Ainsworth, B. E., … Oja, P. (2003). International physical activity questionnaire: 12-country reliability and validity. Medicine and Science in Sports and Exercise, 35(8), 1381–1395. http://doi.org/10.1249/01.MSS.0000078924.61453.FB        [ Links ]

Fernandes, R. A., Christofaro, D. G. D., Casonatto, J., Codogno, J. S., Rodrigues, E. Q., Cardoso, M. L., … Zanesco, A. (2011). Prevalence of dyslipidemia in individuals physically active during childhood, adolescence and adult age. Arquivos Brasileiros de Cardiologia, 97(4), 317–323. http://doi.org/10.1590/S0066-782X2011005000083        [ Links ]

Fernandes, R. A., & Zanesco, A. (2010). Early physical activity promotes lower prevalence of chronic diseases in adulthood. Hypertension Research, 33(9), 926–931. http://doi.org/10.1038/hr.2010.106        [ Links ]

Freedson, P. S., Melanson, E., & Sirard, J. (1998). Calibration of the Computer Science and Applications, Inc. accelerometer. Medicine and Science in Sports and Exercise, 30(5), 777–781. http://doi.org/10.1097/00005768-199805000-00021        [ Links ]

Grimm, E. K., Swartz, A. M., Hart, T., Miller, N. E., & Strath, S. J. (2012). Comparison of the IPAQ-Short Form and accelerometry predictions of physical activity in older adults. Journal of Aging and Physical Activity, 20(1), 64–79.         [ Links ]

Hallal, P. C., Gomez, L. F., Parra, D. C., Lobelo, F., Mosquera, J., Florindo, A. A., … Sarmiento, O. L. (2010). Lessons learned after 10 years of IPAQ use in Brazil and Colombia. Journal of Physical Activity & Health, 7 Suppl 2, S259–264.         [ Links ]

Haskell, W. L., Lee, I.-M., Pate, R. R., Powell, K. E., Blair, S. N., Franklin, B. A., … American Heart Association. (2007). Physical activity and public health: updated recommendation for adults from the American College of Sports Medicine and the American Heart Association. Circulation, 116(9), 1081–1093. http://doi.org/10.1161/CIRCULATIONAHA.107.185649        [ Links ]

IPAQ. (2005). Guidelines for data processing and analysis of the International Physical Activity Questionnaire (IPAQ)—Short and Long Forms. IPAQ. Obtido de http://www.ipaq.ki.se/scoring.pdf        [ Links ]

Johnson-Kozlow, M., Sallis, J. F., Gilpin, E. A., Rock, C. L., & Pierce, J. P. (2006). Comparative validation of the IPAQ and the 7-Day PAR among women diagnosed with breast cancer. The International Journal of Behavioral Nutrition and Physical Activity, 3, 7. http://doi.org/10.1186/1479-5868-3-7        [ Links ]

Matsudo, V. K. R., Matsudo, S. M., Araújo, T. L., Andrade, D. R., Oliveira, L. C., & Hallal, P. C. (2010). Time trends in physical activity in the state of São Paulo, Brazil: 2002-2008. Medicine and Science in Sports and Exercise, 42(12), 2231–2236. http://doi.org/10.1249/MSS.0b013e3181e1fe8e        [ Links ]

Oyeyemi, A. L., Bello, U. M., Philemon, S. T., Aliyu, H. N., Majidadi, R. W., & Oyeyemi, A. Y. (2014). Examining the reliability and validity of a modified version of the International Physical Activity Questionnaire, long form (IPAQ-LF) in Nigeria: a cross-sectional study. BMJ Open, 4(12), e005820. http://doi.org/10.1136/bmjopen-2014-005820        [ Links ]

Poehlman, E. T., Toth, M. J., & Gardner, A. W. (1995). Changes in energy balance and body composition at menopause: a controlled longitudinal study. Annals of Internal Medicine, 123(9), 673–675. http://doi.org/10.7326/0003-4819-123-9-199511010-00005        [ Links ]

Rebelatto, J. R., & Castro, A. P. (2007). Efeito do programa de revitalização de adultos sobre a ocorrência de quedas dos participantes. Brazilian Journal of Physical Therapy, 11(5), 383–389. http://doi.org/10.1590/S1413-35552007000500009        [ Links ]

Siervo, M., & Jebb, S. A. (2010). Body composition assessment: theory into practice: introduction of multicompartment models. IEEE Engineering in Medicine and Biology Magazine: The Quarterly Magazine of the Engineering in Medicine & Biology Society, 29(1), 48–59. http://doi.org/10.1109/MEMB.2009.935471        [ Links ]

Trost, S. G., McIver, K. L., & Pate, R. R. (2005). Conducting accelerometer-based activity assessments in field-based research. Medicine and Science in Sports and Exercise, 37(11 Suppl), S531–543.         [ Links ]

Tudor-Locke, C., Johnson, W. D., & Katzmarzyk, P. T. (2010). Accelerometer-determined steps per day in US children and youth. Medicine and Science in Sports and Exercise, 42(12), 2244–2250. http://doi.org/10.1249/MSS.0b013e3181e32d7f        [ Links ]

Ward, D. S., Evenson, K. R., Vaughn, A., Rodgers, A. B., & Troiano, R. P. (2005). Accelerometer use in physical activity: best practices and research recommendations. Medicine and Science in Sports and Exercise, 37(11 Suppl), S582–588.         [ Links ]

 

Agradecimentos:
Nada a declarar.
Conflito de Interesses:
Nada a declarar.
Financiamento:
Nada a declarar.

Manuscript received June 29th, 2014; Accepted November 12th, 2014

 

* Corresponding author: Department of Physical Education, Univ. Estadual Paulista "Júlio Mesquita Filho", Roberto Simonsen Street, nº 305 – Education Center, Presidente Prudente, 19060-900, São Paulo, Brazil. E-mail: rossifabricio@yahoo.com.br

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons