SciELO - Scientific Electronic Library Online

 
vol.35 número2A intervenção farmacológica e não farmacológica na PHDA: A perspetiva dos profissionais de saúdeCaracterísticas psicométricas da versão portuguesa da Lista de Sintomas Pediátricos em Idade Pré-Escolar para crianças dos 18 aos 60 meses: índice de autoresíndice de assuntosPesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

  • Não possue artigos similaresSimilares em SciELO

Compartilhar


Psicologia

versão impressa ISSN 0874-2049

Psicologia vol.35 no.2 Lisboa dez. 2021  Epub 31-Dez-2021

https://doi.org/10.17575/psicologia.v35i2.1761 

Artigos Originais

“What do you think people feel?” Emotional climate during a pandemic: Adaptation and validation of a scale

“O que achas que as pessoas sentem?” Clima emocional durante a pandemia: Adaptação e validação de escala

Cristina de Sousa1  2  3 
http://orcid.org/0000-0002-2822-4853

João Viseu3  4 

Helena Vinagre4 

Dario Páez5 

Olga Valentim6  7 

1 Higher School of Health Atlântica, Portugal;

2 RECI - Research Unit in Education and Community Intervention, Portugal.

3 ISEIT Almada, Piaget Institute, Portugal.

4 Faculty of Human and Social Sciences, Research Centre for Tourism, Sustainability and Well-being, University of Algarve, Portugal.

5 Department of Social Psychology, University of the Basque Country, Spain.

6 Higher School of Health Ribeiro Sanches, Portugal.

7 Polytechnic Institute of Lusofonia, Portugal.


Abstract

Our study examined the psychometric properties and factor structure of an instrument to assess emotional climate during the COVID-19 pandemic using a sample of 601 Portuguese individuals. Two sub-samples were created, one to perform an exploratory factor analysis (EFA), composed of 300 participants, and the other to conduct a confirmatory factor analysis (CFA), composed of 301 individuals. Two factors were found: positive and negative emotional climate. In the EFA, both factors established a negative and significant correlation. However, the CFA presented a better fit with two independent factors. Reliability analysis indicated acceptable values for both dimensions. There was also evidence of discriminant, convergent, and criterion validity. More negative emotions were perceived regarding the social climate. Results were discussed in the frame of different positive and negative psychosocial variables related to positive and negative emotional climates, as well as positive emotional climate as a resource for collective resilience.

Keywords: Confirmatory factor analysis; Emotional climate; Exploratory factor analysis; Validity; Reliability

Resumo

O nosso estudo avaliou as propriedades psicométricas e estrutura fatorial de um instrumento para medir o clima emocional durante a pandemia de COVID-19 usando uma amostra de 601 indivíduos portugueses. Foram criadas duas subamostras, uma para realizar uma análise fatorial exploratória (AFE), composta por 300 participantes, e outra para fazer uma análise fatorial confirmatória (AFC), composta por 301 indivíduos. Encontraram-se dois fatores: clima emocional positivo e negativo. Na AFE, estes fatores estabeleceram uma correlação negativa. Contudo, a AFC apresentou um melhor ajustamento para dois fatores independentes. A análise de fiabilidade indicou valores aceitáveis para ambas as dimensões. Existiu também evidência de validade descriminante, convergente e de critério. Foram percebidas mais emoções coletivas negativas relativamente ao clima social. Os resultados foram discutidos considerando a relação entre diferentes variáveis psicossociais positivas e negativas e climas emocionais positivos e negativos, bem como considerado o clima emocional positivo como um recurso de resiliência coletiva.

Palavras-chave: Análise Fatorial Confirmatória; Clima emocional; Análise Fatorial Exploratória; Validade; Fidelidade

Emotional Climate

In recent years, several studies have demonstrated the importance of emotions in individual and collective behaviors. Although emotions are perceived individually, they can be experienced not only by the individual, but also collectively, from a social interaction perspective. Emotional climate refers to the mood, collective emotions, evaluations, and affective action tendencies predominant in a social group (Páez et al., 2013). This collective mood and affective state are developed through the social interaction between the members of a group in a particular environment and is based on the predominance of patterns of social behavior, e.g., of rapprochement and intense social contact in positive emotional climate of joy and trust (De Rivera, 1992). It consists in the predominance and relative salience of a set of emotional circumstances. Psychological climate, from this perspective, is defined as the emotional meaning that an individual, attributes to their social environment. In addition, group moods often represent traces of past collective emotions and act as contexts in which they arise (Páez et al., 2013). Finally, the predominance of a pleasant mood is a defining element of positive emotional climate. Organizational social psychology defines emotional climate as a particular form of organizational climate, which refers, specifically, to the collective mood of an organization's members and their attitudes towards their peers and leaders, as well as the organization. From this perspective, climate, although related to organizational culture, differs from it as it is a function of the organization's policies and procedures, rather than the beliefs, values, and assumptions of its members. A positive or healthy climate is characterized by positive emotions, created, and then nurtured by the organizational structure.

According to Techio et al. (2011), the emotional experience, which refers to groups and social processes, means that although there are individual emotions, their distribution and transmission are collective characteristics. Thus, collective forms of emotion, such as the atmosphere and emotional climate, are conceived as specific psychosocial processes due to: (a) their content, as they are composed of beliefs, emotions, and behaviors related to social groups and institutions; (b) their collective origin, i.e., they have a social nature; (c) their sharing and distribution among social groups; and (d) their functions, as their effects are adaptive in the social, not only in the personal sphere. These processes also reflect how an individual thinks that most of the people feel about their ingroups’ situation. From this perspective, people perceive and experience the emotions that are predominant in their belonging group (i.e., ingroup) or in relevant exogroups. This set of basic emotions distributed socially, together with certain social representations about the world and social future, fulfill purposes of social regulation and constitute the emotional climate (De Rivera, 1992). Unlike emotional atmospheres, which depend on group members focusing on a particular event, emotional climate is based on stable relationships between them. Thus, emotional climate would be more stable than an episodic collective emotion, although it would be more flexible than the emotional culture - the affective patterns that characterize a society or organization over long periods of time (De Rivera & Páez, 2007; Páez et al., 2013).

Therefore, the emotional climate is composed of a relatively stable mood and frequent recurrent emotions, perceived, and shared by individuals in a social context; emotions that result from social interaction around traumatic events of a collective nature and that (in)directly affect most members of the society (De Rivera, 1992; Páez et al., 2013). The socio-emotional climate experienced by the members of a society or nation would be characterized by a certain affective tone due to the predominance of some emotions (Hernández et al., 2015; Páez et al., 1997)

According to this framework, people experience both positive and negative emotions. So, emotional climate is the result of a balance between these emotions. For example, if there are higher levels of positive emotions, the emotional climate will be positive (Páez et al., 1997).

Emotional Climate and Psychosocial Context

Emotional climate acts as a psychosocial context that influences behavior (Bar-Tal et al., 2007). Fernández-Dols et al. (2007) have emphasized the importance of socio-emotional climate as a key factor to explain the social order, caused by emotional conventions, which not only categorize the meaning of emotions, but also regulate emotional behaviors. The identification and monitoring of some emotional categories is important to identify changes and avoid compromising social situations (Hernández et al., 2015). Fernández-Dols et al. (2007) have argued that even catastrophic, extremely unusual, unforgettable events produce a diverse range of affective and emotional reactions in any complex, large society. When a collective or a group is threatened by a particular event, individuals experience different emotions depending on their: (a) position in the social structure; (b) personality; and (c) age, i.e., the salient features of their identity. As such, these authors considered that emotional climate should not be defined in a social unanimity, but as grounded on emotional conventions and, specifically, as the way in which emotional conventions prime some categories of emotion in a particular society. When a climate is consolidated, a certain consensus is formed, consequently developing a social representation about the social situation and the norms regarding what should be felt at a given moment (Techio et al., 2011).

Evidence suggests that emotional climate is related to well-being and psychosocial variables. A meta-analysis found that the psychological climate of a working group, characterized as warm and cooperative, and by positive emotions, such as pride, i.e., a concept similar to positive emotional climate, was associated in nine studies with psychological well-being (Parker et al., 2003). A cross-cultural study found that well-being, as a quality-of-life indicator linked to health, was strongly correlated with positive emotional climate and group organizational factors (i.e., positive and innovative leadership, organizational resources, and integration); while occupational stress was found to be negatively associated with positive emotional climate and individual well-being (da Costa et al., 2020). Studies in different communities and countries have revealed that the perception of social problems is associated with low positive emotional climate levels and high negative emotional climate levels (Páez et al., 1997). For instance, positive climate was negatively related to the perception of social problems and positively related to institutional trust in a Mexican sample (Hernández et al., 2015) but to collective self-esteem and national identification in a Peruvian sample (Páez et al., 2013). Finally, negative emotional climate appears to be strongly related to negative affects: stress, low physical and mental health (e.g., depression, anxiety), and frequency of unpleasant events, but not, or to a lower extent, to positive outcomes. Positive emotional climate has been found to correlate with positive affects: positive forms of sociability (e.g., social support and integration), positive collective coping (e.g., demonstrations and collective coping), and frequency of pleasant events, but not, or rarely, to negative outcomes (Páez et al., 2013).

Currently, the world is facing a public health emergency because of the widespread of the coronavirus pandemic (COVID-19). The number of cases and confirmed deaths worldwide has taken on frightening dimensions and requires the adoption of measures with a strong impact on the daily lives of individuals. Social confinement, which reduced physical contact and socialization, in addition to social isolation, led to a severe economic crisis, which increased the rate of bankruptcies and unemployment, as well as leading to a decrease in family income. The COVID-19 pandemic had an impact on individual emotions, mainly due to feelings of fear and insecurity, with consequences on the behavior of individuals, which is important to evaluate and study (Asmundson & Taylor, 2020; Wang et al., 2020). The existence of shared social representations about the virus known as coronavirus in the Americas and some similarities between countries from Southern Europe (i.e., Italy, Spain, and Portugal; see Pizarro et al., 2020) was also identified. That is why it is important to understand how the emotional climate is perceived in Portugal during the pandemic context, what collective emotions the Portuguese think their compatriots have developed, as well as to understand the relationship between the socio-emotional climate and variables both of an individual and social nature (i.e., stress, anxiety, depression, life satisfaction, job satisfaction, resilient coping, perceived organizational support, and work-family conflict).

The Emotional Climate Scale

One of the instruments for assessing emotional climate is the Emotional Climate Scale (ECS) of Páez et al. (1997), used in several studies, especially in traumatic events caused by human actions that affect social groups (Páez et al., 1997; Techio et al., 2011), like terrorism (Páez et al., 2007) or in a revolution (Rimé et al., 2017). This scale has been less used in situations not directly caused by individuals or social groups, such as the pandemic, and, therefore, it seems important to proceed with its translation, adaptation, and validation to the Portuguese context. Our main objective was to address the psychometric properties of a modified version of an emotional climate scale during the COVID-19 pandemic in Portugal. More specifically, we aimed to test the factor validity, convergent validity, discriminant validity, criterion validity, and reliability of this measure. We expected the COVID-19 pandemic to have caused a high level of negative emotional climate, which would be related to negative psychosocial variables, like stress, conflict, and low well-being; while positive emotional climate would be related to well-being and positive organizational variables. Lastly, we expected an asymmetric association: a negative climate would be specifically and more strongly associated with conflict, stress, and social problems; while positive climate would be associated with social belonging, coping, and positive events.

METHOD

Participants

A sample of 601 Portuguese individuals was collected. From this sample, two sub-samples were created, one to perform an exploratory factor analysis (EFA) and the other to conduct a confirmatory factor analysis (CFA). Participants were randomly assigned to these sub-samples.

The EFA sample was composed of 300 participants, 167 (65.7%) females and 103 (34.3%) males, with an average age of 39.74 years old (SD = 13.161; Min.: 18; Max.: 90). Most of the respondents were married or living in common law (55%), had dependent relatives (e.g., children, parents, grandparents, or other relatives) under their responsibility (53.7%), were Portuguese (93.3%), had a bachelor’s degree (26%), and were employed with an indefinite term employment contract (41.7%). Also, respondents (52%) reported that they experienced changes in the way work-related tasks were performed due to the COVID-19 pandemic (e.g., remote work and/or changes in the work schedule). Although in most situations (78%), individuals indicated that organization-related measures were not adopted within the scope of the pandemic, 22% informed that they were in: (a) partial lay-off (5.7%); (b) full lay-off (7.3%); (c) extraordinary family-based support (5%); (d) isolation (0.3%); and (e) extraordinary economic-based support (3.7%).

The CFA sample was composed of 301 individuals, 111 (36.9%) were females and 190 (63.1%) were males, with an average age of 41.69 years old (SD = 9.372; Min.: 19; Max.: 74). Most of the participants (70%) were married or living in common law, had relatives under their responsibility (66.4%), were Portuguese (97.7%), had completed secondary education (54.2%), and worked with an indefinite term type of contract (58.1%) About 64.2% of the employed participants reported changes in their work due to the COVID-19 pandemic, e.g., remote work and/or changes in the work schedule. Lastly, even though 91.7% of the participants indicated that they were not the target of organization-related measures due to the pandemic, 8.3% informed that they were in: (a) partial lay-off (2%) or full lay-off (2%); (b) extraordinary family-based support (2.7%); and (c) extraordinary economic-based support (1.7%).

A summary of the characteristics from both samples, EFA and CFA, is presented in Table 1.

Table 1 Participant’s Characterization (N = 601) 

Note. M = mean value; SD = standard-deviation value; Min. = minimum value; Max. = maximum value; % = percentage.

Variables and Measures

Socio-emotional climate was assessed by the Emotional Climate Scale (Páez et al., 1997), which is composed of 10 items (e.g., The climate or the affective environment of the country is very good.) with a five-point Likert scale (1-Nothing; 5-A lot). The scale is composed of two dimensions: (a) negative emotional climate, which includes items related to basic negative emotions (e.g., fear, anger, sadness); and (b) positive emotional climate, which includes items like joy, hope, so lidarity, trust in the institutions, and tranquility to speak. Since the original version of this measure was in Spanish, the Emotional Climate Scale was first translated into Portuguese by an independent bilingual and bicultural judge. Subsequently, and with the objective of comparing the two versions, a back translation was performed by another bilingual and bicultural independent judge. After comparing these versions, to clarify possible doubts about the wording of the items, the final version of the instrument in Portuguese was obtained. We also included three items in the negative emotional dimension based on a previous content analysis regarding the emotional climate during the Portuguese economic crisis (De Sousa et al., 2014), 1. apprehension, preoccupation; 2. uncertainty; insecurity; and 3. despair, frustration. The final version of the instrument was composed of 13 items. In the development study of this measure, Cronbach’s alpha values equal or higher than .80 were obtained, specifically .80 for the Spanish sample and .85 for the Colombian sample (Páez et al., 1997).

Work and organizational variables. Job satisfaction was assessed by the Short Index of Job Satisfaction (SIJS; Sinval & Marôco, 2020), which has five items (e.g., I feel fairly satisfied with my present job.) with a five-point Likert scale (1-Strongly disagree; 5-Strongly agree). Items three and five must be reversed to calculate the global job satisfaction index. In the adaptation study of the SIJS for Portugal and Brazil, reliability values above .85 (Portugal Composite Reliability = .90 and Brazil Composite Reliability = .87) were registered. (Sinval & Marôco, 2020).

Perceived organizational support was evaluated by the Survey of Perceived Organizational Support (SPOS; Santos & Gonçalves, 2010), which is composed of eight items (e.g., The organization really cares about my well-being.) with a seven-point Likert scale (1-Strongly disagree; 7-Strongly agree). This survey has two dimensions: cognitive and affective organizational support, with four items each. Items two, three, five, and seven were reversed before calculating the cognitive support dimension. Santos and Gonçalves (2010) obtained a global Cronbach’s alpha value of .87.

Conflict and distress. Work-family conflict was measured by the work-family conflict dimension of the Copenhagen Psychosocial Questionnaire (COPSOQ; Silva et al., 2011), which presents three items (e.g., Do your family and friends say that you work too much?) with a five-point Likert scale (1-Never/almost never; 5-All the time). In the adaptation and validation study of the COPSOQ for a Portuguese working sample, a Cronbach’s alpha value of .86 was registered (Silva et al., 2010).

Stress, anxiety, and depression were evaluated by the Depression Anxiety Stress Scale 21-item version (DASS-21; Pais-Ribeiro et al., 2004), which presents 21 items (e.g., I couldn’t seem to experience any positive feeling at all.) with a four-point answer scale (0-Did not apply to me at all; 3-Applied to me very much, or most of the time-Almost always). This scale has three dimensions, stress, anxiety, and depression, with seven items each. In the study of Pais-Ribeiro et al. (2004) Cronbach’s alpha values were higher than .70 (depression α = .85, anxiety α = .74, and stress α = .81).

Coping. Resilient coping was assessed by the Brief Resilient Coping Scale (BRCS; Pais-Ribeiro & Morais, 2010), which has four items (e.g., I look for creative ways to alter difficult situations.) with a five-point Likert scale (1-Does not describe me at all; 5-Describes me well). Pais-Ribeiro and Morais (2010) registered Cronbach’s alpha values below the threshold of .70. Nevertheless, the authors argued that classic reliability coefficients are not adequate for assessing coping dimensions, namely because these constructs are context dependent. Furthermore, this measure only presented four items, reliability coefficients tend to penalize measures with a reduced number of items.

Well-being. Life satisfaction was measured by the Satisfaction with Life Scale (SWLS; Simões, 1992), which has 5 items (e.g., In most ways, my life is close to my ideal.) with a 7-point Likert scale (1-Strongly disagree; 7-Strongly agree). Reliability results for this scale were higher than the threshold of .70 (Simões, 1992).

Positive and negative affects were assessed by the short form of the Positive and Negative Affect Schedule (PANAS; Galinha et al., 2014), which has ten items (e.g., Interested.) with a five-point Likert scale (1-Not at all or very slightly; 5-Extremely). This measure is composed of two dimensions, positive and negative affects, with five items each. Galinha et al. (2014) registered that PANAS had good psychometric characteristics in a Portuguese sample.

The socio-demographic survey had questions regarding the sex, age, marital status, size of the household, existence of dependent family members, number of dependent family members, nationality, academic background, professional status, changes in the work environment due to the COVID-19 pandemic, and existence of organization-related measures resulting from the pandemic context.

Data Collection Procedures

The research protocol presented the objectives of this study, as well as providing indications regarding anonymity and confidentiality standards. The research team ensured that the collected data would only be used for scientific purposes and that participants could withdraw their participation at any time without any prejudice. It was also mentioned that the response to the protocol was voluntary (i.e., there were no rewards, monetary or otherwise, associated with their participation) and there were no risks, for physical and mental health, in answering the selected self-report measures. After reading this information, potential participants had to indicate whether they agreed with it. If they agreed, they were directed to a page to fill out the research protocol. If not, they were informed that they could not participate in the study.

The protocol was developed through an online platform, with the aim of facilitating data collection and dissemination. A cross-sectional design was adopted, and one exclusion criterion was defined: respondents should be 18 years old or more to participate. Initially, the protocol was sent to the personal contacts of the research team and publicized in social media. In a second moment, participants were asked to send the platform link to their personal contacts. In other words, a non-probabilistic sampling technique was followed, namely of convenience and snowball. Data collection occurred between April-23rd-2020 and May-31st-2020.

Data Analysis Procedures

Initially, the 13 items of the Emotional Climate Scale were the target of an EFA, following the assumptions of Field (2018). Thus, the Principal Axis Factoring method was used with an oblique rotation, Direct Oblimin. This rotation was selected because it was expected that the obtained factors would be correlated. To assess the suitability of the sample for an EFA, the Kaiser-Meyer-Olkin (KMO) test was performed. The values of this test must be greater than .70 for the sample to be considered adequate (Kaiser & Rice, 1974). In addition, the KMO value for each of the scale’s items was calculated. According to Kaiser and Rice (1974), values equal or greater than .50 must be registered for an item to be considered. Bartlett’s sphericity test was used to compare the values between the correlation and identity matrices, with a statistically significant value (p < .05) needing to be achieved. Multicollinearity was tested through the determinant of the R matrix; with the obtained value needing to be higher than .00001. The EFA factors were extracted using the Kaiser criterion and the analysis of the scree plot chart. Only items with a factor loading equal or greater than .50 were maintained. Finally, reliability was tested with the Cronbach’s alpha coefficient, with values above .70 being desired.

Subsequently, a CFA was conducted. The first procedure was the evaluation of the multivariate normal distribution. There is no consensus regarding the skewness and kurtosis values that indicate the presence of a distribution of this nature. However, for the maximum likelihood estimation method, the most common method in structural equation modeling (SEM), skewness and kurtosis values of two and seven, respectively, point to the existence of a multivariate normal distribution (Curran et al., 1996). The global fit of the model was based on the following indices: (a) Chi-squared goodness of fit test (χ2), p-values greater than .05 are expected. However statistically significant results (p < .05) may emerge due to the sample size; (b) Goodness of Fit Index (GFI), values between .90 and .95 indicate a good fit and greater than .95 point to a very good fit; (c) Root Mean Square Error of Approximation (RMSEA), values between .05 and .10 reflect an acceptable fit and below .05 suggest a very good fit; (d) Standardized Root Mean Squared Residual (SRMR), values between .05-.08 are acceptable and below .05 are good; (e) Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI), values between .90 and .95 point to a good fit and higher than .95 indicate a very good fit; and (f) χ2/df, values below five are acceptable and below two are good (Byrne, 2010; Hu & Bentler, 1999; Marôco, 2014). The Aikaike Information Criterion (AIC) was also measured, the model with the best fit will be the one with the lowest value.

Validity and reliability were also tested. Regarding validity, a quadripartite evaluation, factor, convergent, discriminant, and criterion, was performed. Factor validity was calculated based on the standardized factor loadings of the indicators of the Emotional Climate Scale, values equal or greater than .50 are desired (Marôco, 2014). To analyze convergent validity, the Average Variance Extracted (AVE) coefficient was used, AVE > .50 indicates the existence of this type of validity (Bagozzi & Yi, 1988; Sharma, 1996). Discriminant validity was assessed through the Fornell and Larcker (1981) criterion, i.e., comparison between the AVE values of each construct and the squared correlation values. To test this type of validity, the PANAS (Galinha et al., 2014) was selected. Criterion validity was calculated using the concepts of stress, anxiety, depression, life satisfaction, job satisfaction, resilient coping, perceived organizational support, and work-family conflict (Pais-Ribeiro et al., 2004; Pais-Ribeiro & Morais, 2010; Santos & Gonçalves, 2010; Silva et al., 2011; Simões, 1992; Sinval & Marôco, 2020). Reliability was evaluated by two coefficients, Cronbach’s alpha and Composite Reliability (CR), with values greater than .70 being desired (Hair et al., 1998).

Results

Exploratory Factor Analysis

The KMO value was .844, which is classified as meritorious by Kaiser and Rice (1974). This demonstrated that the sample could be the target of an EFA. The KMO values for the items ranged between .742 and .897, values higher than the cut-off defined. Bartlett’s sphericity test registered a statistically significant value (p < .05). Lastly, the determinant of the R matrix obtained a result of .016, which indicates that there are no multicollinearity issues. After verifying these assumptions, the obtained factors were analyzed.

According to the Kaiser criterion, two factors, which explained 43.168% of the total variance, were extracted. The first factor explained 29.763% of the variance and the second explained 13.404%. Field (2008) has argued that the Kaiser criterion is reliable when the number of variables is smaller than 30, the sample has 250 or more participants, and the communalities average is equal or greater than .60. In this study, there were 13 indicators (i.e., items) and the sample was composed of 300 individuals. However, the communalities average was .432, i.e., < .60. Thus, the scree plot chart was analyzed. It was found that the best option would be to maintain the two factor structure (see Table 2). The first factor was composed of five items and the second of three, with five items being removed because they presented factor loadings lower than .50. The items of the first factor (α = .879) had a negative nature, whereas the second (α = .682) presented items with a positive nature. Lastly, it was observed that both factors established a negative correlation (r = -.179; p < .01).

Table 2 Factor Loadings of the Items of the Socio-Emotional Scale Extracted after the EFA (N = 300) 

Note. Only items with a factor loading > .50 are presented. M = mean value; SD = standard-deviation value; α = Cronbach’s alpha value.

Confirmatory Factor Analysis

Before the CFA, skewness and kurtosis values were analyzed to observe the existence of a multivariate normal distribution. The obtained values respected the cut-off defined. It was thus possible to conduct a CFA through the maximum likelihood estimation method. Three competing factor structures were tested: (a) model with two-correlated factors (Model 1); (b) model with two independent factors (Model 2); and (c) model with a second-order factor (Model 3). Initially, none of these structures had an acceptable fit (see Table 3).

Table 3 Fit Indexes for the Proposed Factor Structures (N = 301) 

Note. *p < .05; Model 1 = Model with two-correlated factors; Model 2 = Model with two independent factors; Model 3 = Second-order model.

Subsequently, an assessment of the modification indices was performed. A covariance between the errors of the items “The environment or social climate is of sadness, passiveness, and low mood” and “The environment and social climate is of despair and frustration” was created. After this procedure, the respecified models presented an adequate fit (see Table 4). The model that presented a better fit was composed of two independent factors (Model 2).

Table 4 Fit Indexes for the Respecified Factor Structures (N = 301) 

Note. *p < .05; Model 1 = Model with two-correlated factors; Model 2 = Model with two independent factors; Model 3 = Second-order model.

Measurement Model Fit

The next step was the analysis of factor validity. It was found that all items had standardized factor loadings above .50, specifically between .517 and .900 (see Table 5). Regarding convergent validity, the items from the positive emotional climate factor presented an average variance extracted value below the threshold of .50. Furthermore, reliability values for this factor were also below the cut-off. Nevertheless, Bagozzi and Yi (1988) argued that results higher than .60 can be considered acceptable (see Table 5).

Table 5 Multivariate Normal Distribution, Factor Validity, Convergent Validity, and Reliability Values for the Select Factor Structure (N = 301) 

Note. *p < .05; sk = skewness; ku = kurtosis; Std. loadings = standardized factor loadings; Alpha = Cronbach’s alpha; CR = composite reliability; AVE = average variance extracted.

To assess discriminant validity, the Fornell and Larcker (1981) criterion was used, a comparison was performed between the squared correlation values and AVE values for both factors of the Emotional Climate Scale and the two dimensions (positive and negative affects) of the PANAS (Galinha et al., 2014). Before this procedure, a CFA with the items of the PANAS was performed. To achieve an adequate fit, items one (interested) and eight (guilty) were removed, since they presented standardized factor loadings below .50, as well as establishing a covariance between the errors of items six (active) and nine (determined). The model obtained after these changes showed a good fit: χ2 = 42.919 (p < .05); df = 18; GFI = .965; RMSEA = .068; 90% RMSEA = [.042; .094]; SRMR = .0494; CFI = .978; TLI = .966; χ2/df = 2.384. After obtaining this structure, discriminant validity was calculated (see Table 6).

Table 6 Discriminant Validity of the Emotional Climate Scale (N = 301) 

Note. The AVE values are in bold.

There is evidence of discriminant validity, since the AVE values were higher than the squared correlation values.

Criterion validity was also tested and it was expected that: (a) positive emotional climate would establish a positive and statistically significant correlation with positive affects: affective and cognitive organizational support, resilient coping, and job and life satisfaction, and a negative and statistically significant correlation with negative emotions, negative affects: stress, anxiety, depression, and work-family conflict; (b) negative emotional climate would establish a positive and statistically significant correlation with negative affects: stress, anxiety, depression, and work-family conflict and a negative and statistically significant correlation with positive emotions, positive affects: life satisfaction, affective support, cognitive support, resilient coping, and job satisfaction. This analysis was conducted with the whole sample (N = 301) for the correlations with stress, anxiety, depression, life satisfaction, and resilient coping. For the relationships with job satisfaction, perceived organizational support, and work-family conflict, only employed individuals (N = 273) were selected. Through this analysis, it was found that positive emotional climate was positively and significantly correlated (p < .001) with positive affects: life satisfaction, affective organizational support, and cognitive organizational support, and negatively and significantly correlated (p < .001) with work-family conflict. Moreover, positive emotional climate was unrelated with stress, anxiety, depression, and negative affects. Furthermore, negative emotional climate was positively and significantly correlated (p < .001) with negative affects: stress, anxiety, depression, and work-conflict, and negatively and significantly correlated with life satisfaction (p < .05), affective support (p < .05), and job satisfaction (p < .001), but was unrelated to positive emotional climate, which was an unexpected result (see Table 7).

After examining the items and the overall mean values, we found that the participants perceived more negative collective emotions (M = 3.51) than positive collective emotions (M = 3.13). Considering the negative emotional climate dimension, uncertainty, insecurity; and apprehension, preoccupation showed the highest means followed by fear and anxiety.

Table 7 Criterion Validity of the Emotional Climate Scale (N = 301) 

Note. ** p < .001; *p < .05; 1The correlation analysis was only performed with employed individuals (N = 273); On the diagonal of positive emotional climate and negative emotional climate are the Cronbach’s alpha and CR values, respectively. On the diagonal of positive affects (3) to job satisfaction (10) are the Cronbach’s alpha values.

Discussion

According to the obtained results, the adaptation of Emotional Climate Scale (ECS) of Páez et al. (1997), proved to be a valid instrument for the evaluation of emotional climate during the pandemic context in a Portuguese sample. The EFA enabled the identification of a structure composed of two independent factors, grouped in a similar way to that proposed by Páez et al. (1997), although with a reduction in the number of items per factor. The first factor was composed of five items and had a negative nature, the second was composed of three items with a positive nature. It was observed that both factors established a negative and significant correlation. However, the CFA presented a better fit in a model with two independent factors. The final model achieved an acceptable to very good fit (Byrne, 2010; Hu & Bentler, 1999; Marôco, 2014). In this factor solution, all indicators (i.e., items) reached standardized factor loadings higher than .50: between .517 and .900. Regarding convergent validity, in the positive emotional climate dimension the result obtained was below the cut-off value defined by the literature (AVE > .50; Bagozzi & Yi, 1988; Sharma, 1996). This was an unexpected situation. The literature, however, provides possible explanations to justify this situation. According to Valentini and Damásio (2016), the AVE coefficient tends to discriminate models with lower standardized factor loadings or with homogeneous standardized factor loadings. As such, these authors argued that it is not appropriate to establish cut-off values for this coefficient, suggesting that each model should be assessed critically, namely in terms of its characteristics; and then conclude what are the cut-off values that should be adopted. In other words, what Valentini and Damásio (2016) proposed is that there is no rule of thumb regarding the AVE values, this coefficient should be analyzed according to the specificities of each model. Furthermore, there was evidence of discriminant validity, since the AVE values were higher than the squared correlation values of the considered variables. For the positive emotional climate dimension, there was evidence of criterion validity. The correlation coefficients, in terms of direction and statistical significance, achieved the expected results, except for the correlations between positive emotional climate and resilient coping and between positive emotional climate and job satisfaction. In terms of the negative emotional climate dimension, the same was observed, except for the correlation between negative emotional climate and resilient coping.

Regarding reliability, the results for both negative emotional climate dimension coefficients were higher than the threshold of .70 (Hair et al., 1998). This situation was not registered for the positive emotional climate dimension. Both coefficients, Cronbach’s alpha and CR, were influenced by the number of items, i.e., with more indicators, the likelihood of achieving a better result is higher (Valentini & Damásio, 2016). In this case, the fact that this dimension had few items was detrimental. Nevertheless, Bagozzi and Yi (1988) suggested that reliability values higher than .60 are acceptable, contrary to the position of Hair et al. (1998).

The results partially confirmed the expected asymmetric association between negative and positive emotional climate and psychosocial variables. Negative emotional climate is supposedly determined mainly by social problems, conflict, and collective stress; while positive emotional climate is mainly associated with social support, resolution of social problems, participation in altruistic behaviors, and collective constructive forms of coping (Páez et al., 2013). Supporting this hypothesis, negative emotional climate was strongly associated with negative affects: stress and ill-being; while positive emotional climate was associated with well-being and perceived organizational support. Likewise, positive emotional climate was not associated with stress, anxiety, and depression. Work-Family conflict was associated with both emotional climate dimensions, establishing relationships with a similar effect, but with a different sign, suggesting that this psychosocial variable may be relevant because of its impact on the social, family, and work domains.

An unexpected result was the absence of a relationship between positive emotional climate, resilient coping, and job satisfaction. Regarding job satisfaction, contextual aspects, such as the pandemic, may explain this situation. Many individuals suffered a sudden change in their way of working, going from face-to-face work to remote work, often without the necessary conditions and training, which may have hampered their satisfaction levels. The lockdown may also have weakened the relationship between positive emotional climate and constructive forms of coping - but more research is needed to examine these speculative explanations. These results can also be explained by a limitation of this study. The measure was of national emotional climate; while the measures of conflict, perceived organizational support, and stress, anxiety, and depression were of an individual and interpersonal nature. Probably, if organizational climate had been measured, relationships would have been stronger and more consistent with job satisfaction and perceived organizational support measures.

Theoretical and Practical Implications

According to the definition of emotional climate (De Rivera, 1992; Páez et al., 2013), the pandemic context represents a collective traumatic event that affected most members of society, and is characterized by the perception of collective emotions.

However, in the beginning, this event was essentially an unknown threat, since it was not very common in our society. Therefore, our results are congruent with this “unknown threat affect” and represent an important contribution to study contexts characterized by an affective tone of uncertainty, insecurity, apprehension, and preoccupation, like pandemic ones. Our study also contributes to an understanding of the important role that a supportive social system has on the perception of emotional climate, providing some evidence that perceived organizational support and work-family conflict are important resources, pillars, to face this type of “unknown threat affect”.

A practical social relevance of our study is related to the knowledge of what people think others feel, allowing us to assess the emotional collective emotions felt in these types of events, as a psychosocial context that influences behavior (Bar-Tal et al, 2007). This gives us an instrument to prevent risk behaviors associated with insecurity, fear, and anxiety; promote actions with the objective of reducing stress and increase well-being; and also create community actions to develop access to secure information, crucial to reducing uncertainty.

Limitations and Suggestions for Future Research

This study has some limitations worth considering. The first is related to the sample size, which is not representative of the Portuguese context. Furthermore, statistically significant differences were found at the level of sex, marital status, educational background, employment status, and organization-related measures due to COVID-19 between the samples from the EFA and CFA. This aspect may have compromised the obtained results, since both samples were heterogeneous for these socio-demographic characteristics. On the other hand, convergent validity results, measured through the AVE coefficient, for the positive emotional climate dimension were below the cut-off of .50. The AVE has been the target of some criticism, especially because it overestimates models with indicators that present heterogeneous standardized factor loadings and underestimates models with items that have homogeneous standardized factor loadings (Valentini & Damásio, 2016). For this same dimension, reliability values were also below the cut-off defined by the literature: .70. This situation may have occurred because internal consistency coefficients tend to discriminate against constructs or dimensions with a reduced number of items, such as the positive emotional climate dimension, which was measured by three items (Valentini & Damásio, 2016). Moreover, some results, at the criterion validity level, were unexpected. Lastly, the temporal invariance of the validated instrument was not assessed.

Future studies should seek to collect a representative sample of the Portuguese reality and ensure that the different samples used are homogeneous regarding their socio-demographic characteristics. Also, some psychometric characteristics, such as convergent validity, criterion validity, and reliability, should be the target of a careful evaluation to observe whether the results obtained respect the statistical assumptions defined in the literature. It will also be interesting to perform comparative studies, e.g., between different periods of the pandemic, to assess if emotional climate perceptions differ between distinct societal contexts, or between two different countries, e.g., Portugal and Brazil, to verify if there are differences between these countries regarding the perceptions of emotional climate during the pandemic.

Conclusion

The adaptation and validation of the Emotional Climate Scale (Páez et al., 1997) for a Portuguese sample achieved adequate results in terms of validity (factor, convergent, discriminant, and criterion) and reliability. Notwithstanding, there are aspects that should be addressed in future research, namely in terms of convergent validity, criterion validity, and reliability for the positive emotional climate dimension. The results are also congruent with this “unknown threat affect” that the pandemic represents, and the socio-emotional perceptions characterizing the answers to the question “What do you think people feel?” highlights this threat.

As social resources, perceptions of positive or negative support by organizations in this crisis also seemed to be important in the assessment of emotional climate. Furthermore, the role of local institutions, as work organizations, during this type of health and economic crisis, may be determinant to prevent negative outcomes, such as stress, anxiety, depression, work-family conflict and improve positive outcomes, e.g., life and job satisfaction. According to these results, organizational variables should be considered in future studies as moderating or mediating variables.

References

Asmundson, G. J. G., & Taylor, S. (2020). Coronaphobia: Fear and the 2019-nCoV outbreak. Journal of Anxiety Disorders, 70, 102-196. https://doi.org/10.1016/j.janxdis.2020.102196 [ Links ]

Bagozzi, R., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74-94. http://doi.org/10.1007/BF02723327 [ Links ]

Bar-Tal, D., Halperin, E., & de Rivera, J. (2007). Collective emotions in conflict situations: Societal implications. Journal of Social Issues, 63(2), 441-460. https://doi.org/10.1111/j.1540-4560.2007.00518.x [ Links ]

Bouchat, P., Metzler, H., & Rimé, B. (2020). Crise et pandémie: Impact émotionnel et psychosocial du confinement. Journal des Psychologues, 380(8), 14-20. https://doi.org/10.3917/jdp.380.0014 [ Links ]

Byrne, B. (2010). Structural equation modeling with AMOS: Basic concepts, application, and programing (2nd Ed.). Routledge. [ Links ]

Curran, P., West, S., & Finch, J. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29. https://doi.org/10.1037/1082-989X.1.1.16 [ Links ]

Da Costa, S., Martínez-Moreno, E., Díaz, V., Hermosilla, D., Amutio, A., Padoan, S., Méndez, D., Etchebehere, G., Torres, A., Telletxea, S., & García-Mazzieri, S. (2020). Belonging and social integration as factors of well-being in Latin America and Latin Europe organizations. Frontiers in Psychology, 11, 604412. https://doi.org/10.3389/fpsyg.2020.604412 [ Links ]

De Rivera, J. (1992). Emotional climate: Social structure and emotional dynamics. In K. T. Strongman (Ed.), International review of studies on emotion, Vol. 2 (pp. 197-218). John Wiley & Sons. [ Links ]

De Rivera, J., & Páez, D. (2007). Emotional climate, human security, and cultures of peace. Journal of social Issues, 63(2), 233-253. https://doi.org/10.1111/j.1540-4560.2007.00506.x [ Links ]

De Sousa, C., Alves, M., Boazinha, I., Guerra, G., Kaloga, O., Neves, C., Patronilho, A., Pimenta, A. C., Silva, D., Teixeira, F., Vieira, I., & Gonçalves, S. (2014). Emotional climate and economic crisis in Portugal: A comparative study between employees and unemployed people. In C. de Sousa & A. M. Oliveira (Eds.), Proceedings of the 14th European Conference on Facial Expression: New Challenges for Research (pp. 129-136). IPCDVS. [ Links ]

Fernández‐Dols, J. M., Carrera, P., De Mendoza, A. H., & Oceja, L. (2007). Emotional climate as emotion accessibility: How countries prime emotions. Journal of Social Issues, 63(2), 339-352. https://doi.org/10.1111/j.1540-4560.2007.00512.x [ Links ]

Field, A. (2018). Discovering statistics using IBM SPSS Statistics. SAGE. [ Links ]

Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39-50. https://doi.org//10.2307/3151312 [ Links ]

Galinha, I., Pereira, C., & Esteves, F. (2014). Versão reduzida da escala portuguesa de afeto positivo e negativo - PANAS VRP: Análise fatorial confirmatória e invariância temporal. Revista PSICOLOGIA, 28(1), 53-65. https://doi.org/10.17575/rpsicol.v28i1.622 [ Links ]

Hair, J., Anderson, R., Tatham, R., & Black, W. (1998). Multivariate data analysis. Prentice Hall. [ Links ]

Hernández, G. R., Lugo, S. J., Calderón, K. C., & Pérez, A. F. (2015). Propiedades psicotécnicas de la Escala de Clima Emocional en habitantes del Estado de México. Revista Mexicana de Investigación en Psicología, 7(1), 32-40. [ Links ]

Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. https://doi.org//10.1080/10705519909540118 [ Links ]

Kaiser, H., & Rice, J. (1974). Little jiffy, mark IV. Educational and Psychological Measurement, 34, 111-117. https://doi.org//10.1177/001316447403400115 [ Links ]

Marôco, J. (2014). Análise de equações estruturais: Fundamentos teóricos, software & aplicações. Report Number. [ Links ]

Páez, D., Basabe, N., Ubillos, S., & González‐Castro, J. L. (2007). Social sharing, participation in demonstrations, emotional climate, and coping with collective violence after the March 11th Madrid bombings. Journal of Social Issues, 63(2), 323-337. https://doi.org/10.1111/j.1540-4560.2007.00511.x [ Links ]

Páez, D., Espinosa, A., & Bobowik, M. (2013). Emotional climate: How is it shaped, fostered, and changed? In D. Hermans, B. Rimé & B. Mesquita (Eds.), Changing Emotions (pp. 113-119). PsychologyPress. [ Links ]

Páez, D., Ruiz, J. I., Gailly, O., Kornblit, A. L., & Wiesenfeld, E. (1997). Clima emocional: Su concepto y medicion mediante una investigacion transcultural. Revista de Psicologia Social, 12 (1), 79-98. https://doi.org/10.1174/021347497320892045 [ Links ]

Pais-Ribeiro, J., Honrado, A., & Leal, I. (2004). Contribuição para o estudo da adaptação portuguesa das Escalas de Ansiedade, Depressão e Stress (EADS) de 21 itens de Lovibond e Lovibond. Psicologia, Saúde & Doenças, 5(2), 229-239. [ Links ]

Pais-Ribeiro, J.L., & Morais, R. (2010). Adaptação portuguesa da escala breve de coping resiliente. Psicologia, Saúde & Doenças, 11(1), 5-13. [ Links ]

Parker, C., Baltes, B. B., Scott, A., Young, S. A., Wuff, J. E., Altman, R. A., Lacost, H. A., & Roberts, J. E. (2003). Relationships between Psychological Climate Perceptions and Work Outcomes: A Meta-Analytic Review. Journal of Organizational Behavior, 24(4), 389-416. https://doi.org/10.1002/job.198 [ Links ]

Pizarro, J. J., Cakal, U., Méndez, L., Da Costa, S., Zumeta, L. N., Gracia-Leiva, M., Nekave, B., Navarro-Carrillo, G., Cazan, A-M., Keshavarzi, S., López-López, W., Yahiiaiev, I., Alzugaray-Ponce, C., Villagrán, L., Moyano-Diaz, E., Petrovic, N., Mathias, A., Techio, E.M., Wlodarczyk, A., … Cavalli, S. (2020). Tell me what you are like and I will tell you what you believe in: Social representations of COVID-19 in the Americas, Europe and Asia. Papers on Social Representations, 29(2), 2.1-2.38. [ Links ]

Rimé, B., Yzerbyt, V., & Mahjoub, A. (2017). Perception of emotional climate in a revolution: Test of a multistage theory of revolution in the Tunisian context. British Journal of Social Psychology, 56(4), 633-654. https://doi.org/10.1111/bjso.12204 [ Links ]

Santos, J., & Gonçalves, G. (2010). Contribuição para a adaptação portuguesa da escala de Percepção de Suporte Organizacional de Eisenberger, Huntington, Hutchison e Sowa (1986). Laboratório de Psicologia, 8(2), 213-223. https://doi.org/10.14417/lp.642 [ Links ]

Sharma, S. (1996). Applied multivariate techniques. John Wiley and Sons Inc. [ Links ]

Silva, C., Amaral, V., Pereira, A., Bem-haja, P., Pereira, A., Rodrigues, V., Cotrim, T., Silvério, J., & Nossa, P. (2011). Copenhagen Psychosocial Questionnaire (COPSOQ): Portugal e países Africanos de língua oficial Portuguesa. Análise Exacta. [ Links ]

Simões, A. (1992). Ulterior validação de uma escala de satisfação com a vida (SWLS). Revista Portuguesa de Pedagogia, 26, 503-515. [ Links ]

Sinval, J., & Mâroco, J. (2020). Short index of job satisfaction: Validity evidence from Portugal and Brazil. PLoS One, 15, e0231474. https://doi.org/10.1371/journal.pone.0231474 [ Links ]

Valentini, F., & Damásio, B. (2016). Average variance extracted and composite reliability: Reliability coefficients. Psicologia: Teoria e Pesquisa, 32(2), 1-7. http://dx.doi.org/10.15689/ap.2018.1704.6.06 [ Links ]

Wang, C., Pan, R., Wan, X., Tan, Y., Xu, L., Ho, C. S., & Ho, R. C. (2020). Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. International Journal of Environmental Research and Public Health, 17(5), 1729. http://doi.org/10.3390/ijerph17051729 [ Links ]

Techio, E., Zubieta, E., Páez, D., De Rivera, J., Rimé, B., & Kanyangara, P. (2011). Clima Emocional y violencia colectiva: estado de la cuestión e instrumentos de medición. In D. Páez, C. Martín-Beristain, J. L. Gonzalez & J. de Rivera (Eds.). Superando la violencia colectiva y construyendo una cultura de paz (pp. 99-148). Fundamentos. [ Links ]

Received: March 29, 2021; Accepted: August 10, 2021

Correspondence address Cristina de Sousa, ISEIT, Avenida Jorge Peixinho, nº30,2805-059 Almada. Portugal. E-mail: c.m.sousa@netcabo.pt

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License