Introduction
Analysis of variance (ANOVA) is a statistical method that evaluates the potential differences among a group of means (Urdan 2017). The dependent variable is often at the scale-level, whereas the independent variable is at the nominal-level and may have two or more categories. The purpose of this section is to provide a brief literature review of ANOVA and present the findings of ANOVA testing about whether there is a relationship between respondents’ views on various management issues based on job levels.
Literature Review
According to Pyrczak (2016), the ANOVA test was developed in 1918 by Ronald Fisher. It is an extension of the t and the z test whose limitation is the inability to tackle nominal level variables with two categories. This test may also be referred to as the Fisher analysis of variance (Rouder et al. 2016). Since its inception, ANOVA has found widespread use among students and researchers. The research design often determines the use of ANOVA. Three ways of ANOVA are possible: one-way ANOVA, two-way ANOVA, and N-way ANOVA (Roberts & Russo 2014).
A one-way ANOVA is applied when there is only one independent variable. For example, one-way ANOVA can be used to examine the math scores of high school students in a school district. On the other hand, a two-way ANOVA is performed when there is a need to compare two independent variables (Curtis et al. 2015). Expanding the initial example provided, a two-way ANOVA can assess the differences in math scores (the dependent variable) by the school (the first independent variable) and gender (the second independent variable). Consequently, it is possible to study the connection between the two independent variables by looking at interactions. Interactions reveal the uneven distribution of variation across all groups of the independent variables. For instance, males may have higher math scores overall compared to females. However, this difference could be greater in private schools than public schools. Another term used to identify two-way ANOVA is factorial ANOVA. Complex studies may compel a researcher to use more than two dependent variables. In this case, an N-way ANOVA is used, where N refers to the number of independent variables (Roberts & Russo 2014). For example, the math scores of high school students can be assessed by the school, gender, ethnicity, and age simultaneously.
Since ANOVA entails comparing means, there is a need to formulate null and alternative hypotheses. The standard null hypothesis for an ANOVA test is that there is no significant difference among groups (Curtis et al. 2015). The alternative hypothesis, conversely, assumes that at least one significant difference exists among the groups.
After formulating the hypotheses, the researcher should test the assumptions of ANOVA. The next step entails computing the F-ratio and the related probability value, which is referred to as the p-value. The null hypothesis is rejected if the p-value linked to the F is smaller than the established level of significance, which could be 0.05 or 0.01. Rejecting the null hypothesis implies that the alternative hypothesis is supported and that the means of all the groups are unequal. Subsequently, the researcher needs to conduct post hoc tests to identify the groups that differ from each other. Post hoc tests are t-tests that analyse mean differences between groups. Several post hoc tests exist, including Dunnet, Scheffe, Bonferroni, and Tukey tests (Kucuk et al. 2016). These tests reduce the chances of type I errors, which is the erroneous rejection of the null hypothesis (Kucuk et al. 2016).
Two main factors are used to determine whether ANOVA is suitable for analysis. The first factor is the level of measurement of the variables. The dependent variable must be a continuous level of measurement, which could be interval or ratio). On the other hand, the independent variables must be categorical, which could be nominal or ordinal. Since ANOVA is a parametric test, it is guided by three assumptions and has some assumptions. The first assumption is that the data are normally distributed. The second assumption of ANOVA is that the variances are homogeneous. This assumption implies that the variance between the groups should be more or less equal. The third assumption of ANOVA is that the observations are independent of each other. Various tests can be used to test these assumptions. For example, Levene’s test or the Brown-Forsythe Test can be used to test the assumption of homogeneity of variance. Similarly, the normality of the distribution can be tested using skewness and kurtosis, histograms, and tests such as the Kolmogorov-Smirnov or Shapiro-Wilk (Roberts & Russo 2014). The study design is useful in determining the assumption of independence. Nevertheless, researchers should be cautious and keep an eye open for irrelevant or confounding variables.
The main strength of ANOVA as a method of data analysis is that it is a robust procedure regarding contraventions of the assumption of normality. Literature published in the 1950s and earlier stated that the F-tests used in ANOVA were not robust following contraventions of the assumption that the populations of the variables follow a normal distribution, especially for unbalanced scenarios and small alpha (α) levels (Field & Wilcox 2017). It was also believed that violating the assumption of equal variances led to drastic type I errors. However, studies conducted by Donaldson in the 1960s revealed that the F-test was conservative despite small divergences from the assumptions of equal variance and normality, which did not affect the overall significance substantially (Field & Wilcox 2017). Additionally, these effects reduce with an increase in the sample size. This realization has increased the popularity of ANOVA as a statistical method of analysis.
Research Hypotheses
Based on the principles and assumptions of ANOVA as explained in the above sub-section, the researcher formulated the hypotheses indicated in Table 1. The hypotheses were grouped based on the four main factors being investigated. The details of each analysis are explained in the following sections.
Table 1. Table of research hypotheses
Analysis of Variance of Determinants of Risk Governance Framework Factors related to Job Level Issues
Analysis using ANOVA was performed to determine if there were any significant differences between the respondent’s perceptions of determinants of risk governance framework factors. A total of 10 factors were tested, including “Strategy”, “Risk Appraisal and Insight”, “Risk Decision and Process Implementation”, “Risk Management and Governance”, “Review Risk Development and Decision”, “Risk Communication”, “Risk Culture”, “Financial and Technical Capacity”, “Risk Appetite”, and “Risk Ownership” between 3 groups based on their job levels (employee, middle management, and top management). A total of 10 hypotheses were tested (1. H0 1: β1 = 0 or ≠ 0; 2. H0 2: β2 = 0 or ≠ 0; 3. H0 3: β3 = 0 or ≠ 0; H0 4: β4 = 0 or ≠ 0; 5. H0 5: β5 = 0 or ≠ 0; 6. H0 6: β6 = 0 or ≠ 0; H0 7: β7 = 0 or ≠ 0; 8. H0 8: β8 = 0 or ≠ 0; 9. H0 9: β9 = 0 or ≠ 0; 9. H0 10: β10 = 0 or ≠ 0). An ANOVA analysis was conducted to justify the statistical differences of the groups’ responses in each category. The hypothesis test was computed using SPSS software at a significance level of 0.05. The F-statistic and the p-values were observed.
Analysis of Variance of Strategy Related to Job Level
Respondents were asked to respond to 9 items by indicating the likelihood of the existence of those factors in their organization. The results indicated in Table 1 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 9 factors tested. Therefore, there was no significant difference between respondents’ perceptions of strategy based on job levels. Therefore, the H0 hypothesis was accepted for the causes of variation related to strategy. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Risk Appraisal and Insight Related to Job Level
Respondents were asked to respond to 10 items by indicating the likelihood of the existence of those factors in their organization. The results indicated in Table 2 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 10 factors tested. Therefore, there was no significant difference between respondents’ perceptions of risk appraisal and insight based on job levels. Therefore, the H0 hypothesis was accepted for the causes of variation related to risk appraisal and insight. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Risk Decision and Process Implementation Related to Job Level
Respondents were asked to respond to 4 items by indicating the likelihood of the existence of those factors in their organization. The results indicated in Table 3 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 4 factors tested. Therefore, there was no significant difference between respondents’ perceptions of risk decision and process implementation based on job levels. Therefore, the H0 hypothesis was accepted for the causes of variation related to risk decision and process implementation. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Risk Management and Governance Related to Job Level
Respondents were asked to respond to 19 items by indicating the likelihood of the existence of those factors in their organization. The results indicated Table 4 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 19 factors tested. Therefore, there was no significant difference between respondents’ perceptions of risk management and governance based on job levels. Therefore, the H0 hypothesis was accepted for the causes of variation related to risk management and governance. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Review Risk Development and Decision Related to Job Level
Respondents were asked to respond to 10 items by indicating the likelihood of the existence of those factors in their organization. The results indicated Table 5 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 10 factors tested. Therefore, there was no significant difference between respondents’ perceptions of review risk development and decision based on job levels. Therefore, the H0 hypothesis was accepted for the causes of variation related to review risk development and decision. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Risk Communication Related to Job Level
Respondents were asked to respond to 12 items by indicating the likelihood of the existence of those factors in their organization. The results indicated Table 6 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 12 factors tested. Therefore, there was no significant difference between respondents’ perceptions of risk communication based on job levels. Therefore, the H0 hypothesis was accepted for the causes of variation related to risk communication. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Risk Culture Related to Job Level
Respondents were asked to respond to 8 items by indicating the likelihood of the existence of those factors in their organization. As shown in Table 2, 2 out of 8 factors tested showed significant differences between respondents’ perceptions of risk culture variation based on job levels. Thus the null hypothesis was refuted. To find out the statistical difference between the views of respondents’ about factor RCU5 “existence of a process for risk culture audit”, the results in Table 2 showed that F = 3.636 with a p-value = 0.030. With regard to factor RCU8 “existence of formal training of fraud risk awareness and ethical culture”, the result in Table 2 showed that F = 3.338 with p = 0.040, which was lower than the previous factor. The findings on the remaining 6 factors that were insignificant are included in Table 7 of Appendix A.
Additional examination of the Tukey HSD post hoc multiple comparison tests with regard to factor RCU5 showed that there was a significant difference in the views of employees and top management (p = 0.023) regarding the existence of a process for risk culture audit in the organization (Table 3). Similarly, the Tukey HSD test showed that there was a significant difference in the views of employees and top management (p = 0.034) regarding the existence of formal training of fraud risk awareness and ethical culture.
Table 3. Post hoc test for factor RCU5 and RCU8
Analysis of Variance of Financial and Technical Capacity Related to Job Level
Respondents were asked to respond to 5 items by indicating the likelihood of the existence of those factors in their organization. The results indicated Table 8 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 5 factors tested. Therefore, there was no significant difference between respondents’ perceptions of financial and technical capacity based on job levels. Therefore, the H0 hypothesis is accepted for the causes of variation related to financial and technical capacity. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Risk Appetite Related to Job Level
Respondents were asked to respond to 9 items by indicating the likelihood of the existence of those factors in their organization. The results indicated Table 9 in Appendix A under the column ‘Sig.’ show p-values greater than 0.05 for the 9 factors tested. Therefore, there was no significant difference between respondents’ perceptions of risk appetite based on job levels. Therefore, the H0 hypothesis is accepted for the causes of variation related to risk appetite. In this case, there was no need for further tests to determine the difference in specific means between the respondents.
Analysis of Variance of Ownership Factors Related to Job Level
Respondents were asked to respond to 7 items by indicating the likelihood of the existence of those factors in their organization. An ANOVA test was performed to determine if there were any significant differences between the respondent’s perceptions of ownership factors related to the job level. Table 4 indicated that there was a significant difference between respondents’ perceptions of 1 out of the 7 ownership factors. The difference was significant on factor O7 “existence of a third-party professional service provider for risk management activities” (F = 4.008, p = 0.021). There was no significant difference in the views of respondents on the other 6 factors as indicated in Table 10 of Appendix A. Thus, there was a need to conduct additional tests on this factor to identify where the differences existed in factor O7.
Additional examination of the Tukey HSD post hoc multiple comparison tests with regard to factor O7 showed that there was a significant difference in the views of employees and top management (p = 0.015) regarding the likelihood of the existence of a third-party professional service provider for risk management activities in the organization (Table 5).
Table 5. Post hoc test for factor O7
Based on the findings of the analysis, only 3 out of the 93 determinants of “Risk Governance Framework” factors related to job level issues were significant. Therefore, the null hypothesis was rejected, leading to the conclusion that there is at least one statistically significant difference between the respondents’ views on determinants of risk governance framework factors related to job level issues. Significant differences existed in one ownership factor (O7 “existence of third-party professional service provider for risk management activities”) and two risk culture factors. However, the contribution of these factors was minimal.
Analysis of Variance of Risk-Based Audit and the Success of Projects Factors Related to Job Level
An ANOVA test was performed to determine if there were any significant differences between the respondent’s perceptions of risk-based audit and the success of projects related to job level. Causes of variation related to risk-based audit and the success of projects was were examined where the 11th hypothesis was tested (11. H0 11: β11 = 0 or ≠ 0). The hypothesis test was done at a significance level of 0.05. The F-statistic and the p-values were observed.
A total of 28 factors was examined based on 3 job levels (employee, middle management, and top management). Out of these, only 5 were significant: RG1 “the achievement of strategy objectives”, RG2 “delivering projects on time and budget”, RG8 “fewer surprises and crisis in projects”, RG9 “more focus on efficiency of project phases”, and RG14 “better organizational readiness”. Table 6 indicated that there was a significant difference between respondents’ perceptions of RG1 based on job levels even though the difference was not highly significant (F = 3.115, p = 0.048). There was a significant difference between respondents’ perceptions of RG2 based on job levels (F = 4.465, p = 0.014). The respondents’ perceptions of RG8, and RG9 were statistically significant at (F = 3.600, p = 0.031) and (F = 3.455, p = 0.035) respectively. Similarly, there was a significant difference between the respondents’ perceptions of RG14 based on job levels even though the difference was not highly significant (F = 3.090, p = 0.049). There was no significant difference in 23 out of 28 factors as indicated in Table 11 in Appendix A.
The statistically significant findings in Table 6 necessitated the performance of post hoc tests to determine the specific groups where significant differences in perceptions occurred. Therefore, Tukey’s HSD post hoc tests were done with regard to factors RG1, RG2, RG8, RG9, and RG14. The findings are summarized in Table 7 where the findings of the column labelled Sig. were used to identify the differences. P-values ˂0.05 were useful in identifying the significant differences. It was noted that:
- For RG1 “the achievement of strategy objectives”, there was a significant difference in the views of employees and middle management regarding the likelihood of achievement of strategy objectives in helping the organization achieve its goals. However, this difference was not highly significant (p = 0.047). Despite the low significance, this finding indicated that the attainment of objectives influences, though its usefulness may be low compared to other risk-based audit processes.
- For RG2 “delivering projects on time and budget”, there was a significant difference in the views of employees and middle management (p = 0.013) regarding the likelihood of delivering projects on time and budget in contributing to helping the organization achieve its goals. This difference could be attributed to the fact that employees report directly to middle-level management and the two parties are likely to engage in discussions concerning the meeting of deadlines to deliver timely projects.
- For RG8 “fewer surprises and crisis in projects”, significant differences in opinions regarding the likelihood of fewer surprises and crisis in projects contributing to the attainment of project objectives were observed between employees and middle management (p = 0.023).
- For RG9 “more focus on efficiency of project phases”, there was no significant difference on the respondents’ regarding the likelihood of increased focus on the efficiency of projects’ phases on realizing organizational objectives between the 3 job levels.
- For RG14 “better organizational readiness”, there was a significant difference in the views of employees and middle management (p = 0.013) regarding the likelihood of enhanced organizational readiness in helping an organization achieve its project objectives.
For the other 23 factors, there was no need to conduct post hoc tests because the factors were not significant. Overall, the ANOVA result for “risk-based audit and the success of projects factors” based on job levels indicated that there were statistically significant differences between the respondents’ perceptions of 5 out of 28 factors tested. Therefore, the null hypothesis was rejected.
Table 7. Post hoc test for factors RG1, RG2, RG8, RG9, and RG14
Analysis of Variance of the Impact of Negative Events of Projects Related to Job Level
An ANOVA test was conducted to determine if there were any significant differences between the respondent’s perceptions of the impact of negative events of projects related to the job level. A total of 10 factors was considered based on 3 job levels (employee, middle management, and top management). Causes of variation related to the impact of negative events of projects factors were examined where the 12th hypothesis was tested (12. H0 12: β12 = 0 or ≠ 0). The hypothesis test was done at a significance level of 0.05. The F-statistic and the p-values were observed. Respondents were asked to rate the occurrence of 10 negative events of projects in the organization. These events included “experiencing schedule delays”, “cost overrun”, “lack of control over the projects phases”, “past project failures”, “the failure of governance model to manage key projects”, “the existence of unresolved issues and disputes”, “a lack of independent monitoring of progress”, “a lack of reporting to board and executives”, “failure to achieve the business objectives”, and “a loss of opportunity cost of doing the wrong projects”.
Table 11 in Appendix A indicated that there was no significant difference between respondents’ perceptions of the 10 factors tested based on job levels. The opinions of respondents on the impact of negative events of projects did not differ significantly, which implied that all employees were in agreement regarding the incidence of specific negative events in the organization. This agreement is a good indicator because it implies that all members of the organization are informed about the ongoing at their workplace with regard to the incidence of negative project activities. There was no need for additional post hoc analyses since the results were insignificant. Therefore, there was sufficient evidence to reject the null hypothesis and conclude that there is no statistically significant difference between the respondents’ views on the impact of negative events of projects factors related to job level issues.
Analysis of Variance of Internal Audit Function in Overseeing Risk Management Related to Job Level
An ANOVA test was conducted to determine if there were any significant differences between the respondent’s perceptions of internal audit function in overseeing risk management related to job level. Causes of variation related to internal audit function in overseeing risk management were examined where the 13th hypothesis was tested (13. H0 13: β13 = 0 or ≠ 0). The hypothesis test was done at a significance level of 0.05. The F-statistic and the p-values were observed. Respondents were asked to rate the importance of various audit factors in the management of organisation risk by choosing one out of five options. A total of 11 factors was tested out of which the opinions of the respondents on 6 factors were significant based on 3 job levels (employee, middle management, and top management). These factors were IAF1 “Providing independent assurance on risk management processes”, IAF6 “Providing assurance through written reports covering how key risks are managed”, IAF8 “Participating in setting the organization’s risk appetite”, IAF9 “Developing the organizational policies for its risk management processes”, IAF10 “Developing risk management strategy for board approval”, and IAF11 “Implementation risk responses on management’s behalf”.
Table 9 indicated that there was a significant difference between respondents’ perceptions of IAF1 based on job levels (F = 3.934, p = 0.022). There was a significant difference between respondents’ perceptions of IAF6 based on job levels (F = 4.274, p = 0.016). These differences could be attributed to differences in work experience, educational level, or age. Respondents’ perceptions of IAF8, IAF9, IAF10, and IAF11 were statistically significant at (F = 3.394, p = 0.037), (F = 3.753, p = 0.026), (F = 4.516, p = 0.013), and (F = 5.087, p = 0.008) respectively.
Table 9. ANOVA for audit function in overseeing risk management related to job level
Having rejected the null hypothesis, it was necessary to determine where the differences were observed using Tukey’s HSD post hoc tests with regard to Factors IAF1, IAF6, IAF8, IAF9, IAF10, and IAF11. In Table 10, it was noted that:
- For IAF1 “Providing independent assurance on risk management processes”, there were differences in the respondents’ opinions regarding the importance of providing independent assurance on risk management processes for risk management between top management and middle management (p = 0.020).
- For IAF6 “Providing assurance through written reports covering how key risks are managed”, there were significant differences (p = 0.012) between the perceptions of top management and employees regarding the regarding the importance of providing assurance through written reports covering how key risks are managed on risk management.
- For IAF8 “Participating in setting the organization’s risk appetite”, there was a significant difference between the perceptions of middle management and top management regarding the importance of participating in setting the organization’s risk appetite on risk management (p = 0.045). Nevertheless, this difference was not highly significant. However, it indicated that participation in creating an organization’s risk appetite was a useful audit function in risk management.
- For IAF9 “Developing the organizational policies for its risk management processes”, there was a significant difference in the perceptions of middle and top management regarding the value of developing the organizational policies for its risk management processes in audit functions (p = 0.030).
- For IAF10 “Developing risk management strategy for board approval”, there was a significant difference between the perceptions of top management and employees on the importance of developing risk management strategy for board approval as an audit function (p = 0.011).
- For IAF11 “Implementation risk responses on management’s behalf”, there was a significant difference between the perceptions of top management and employees on the importance of implementing risk responses on management’s behalf as an audit function (p = 0.016). A significant difference based on this factor was also noted between middle and top management (p = 0.019). This observation indicated that diverse opinions existed regarding the importance of implementing risk responses in an organization. Therefore, there was a need to conduct additional investigations regarding this factor.
Table 10. Post hoc test – factor IAF1, IAF6, IAF8, IAF9, IAF10, and IAF11
The insignificant factors are indicated in Table 12 of Appendix A. Since there were significant differences in respondents’ perceptions in 6 out of the 11 factors examined, there was sufficient evidence to reject the null hypothesis and conclude that there is a statistically significant difference between the respondents’ views on audit function in overseeing risk management related to job level issues.
Summary
This chapter presented the data collected using one-way ANOVA to analyse the statistical differences among the groups’ responses. SPSS software and a significance level of 0.05 were used for this analysis. The testing of hypotheses involved calculations at α = 0.05 and observation of the F-statistic and the p-value to determine significant differences. The observed differences were considered significant in cases where p ˂ 0.05. On the other hand, the differences were considered insignificant when p ≥ 0.05. In addition, Tukey’s HSD post hoc tests were used to find the significant differences between the respondents’ perceptions about the factors under investigation. The results showed that there was a statistically significant difference in respondents’ views in 14 out 142 factors. There were statistically significant variations in 3 out of the 4 areas where causes of variation were investigated.
References
Curtis, MJ, Bond, RA, Spina, D, Ahluwalia, A, Alexander, S, Giembycz, MA, Gilchrist, A, Hoyer, D, Insel, PA, Izzo, AA & Lawrence, AJ 2015, ‘Experimental design and analysis and their reporting: new guidance for publication in BJP’, British Journal of Pharmacology, vol. 172, no. 14, pp. 3461-3471.
Field, AP & Wilcox, RR 2017, ‘Robust statistical methods: a primer for clinical psychology and experimental psychopathology researchers’, Behaviour Research and Therapy, vol. 98, pp. 19-38.
Kucuk, U, Eyuboglu, M, Kucuk, HO & Degirmencioglu, G 2016, ‘Importance of using proper post hoc test with ANOVA’, International Journal of Cardiology, vol. 209, p. 346.
Pyrczak, F 2016, Making sense of statistics: a conceptual overview, 6th edn, Routledge, London.
Roberts, M & Russo, R 2014, A student’s guide to analysis of variance, Routledge, London.
Rouder, JN, Engelhardt, CR, McCabe, S & Morey, RD 2016, ‘Model comparison in ANOVA’, Psychonomic Bulletin & Review, vol. 23 no. 6, pp. 1779-1786.
Urdan, T C 2017, Statistics in plain English, 4th edn, Routledge, London.
Appendices
Appendix A
Table 1. Strategy
Table 2. Risk appraisal and insight
Table 3. Risk decision and process implementation
Table 4. Risk management and governance related to job level
Table 5. Risk development and decision related to job level
Table 6. Risk communication related to job level
Table 7. Risk culture
Table 8. Financial and technical capacity
Table 9. Risk appetite
Table 10. Ownership factors
Table 11. Risk based audit process on success of the projects
Table 12. Impact of negative events of projects
Table 13. The role of internal audit function in risk management.