Vinita Malik1, Sukhdip Singh2

1Information Scientist, Central University of Haryana , Mahendergarh, Haryana,123029, India

2Deenbandhu Chhoturam University of Science and Technology , Murthal  , Haryana, 131039,India


Corresponding author Email:



The present research effort analyses various application areas of risk oriented testing and indentifies the gaps from the past .We need risk oriented testing not only for identifying risks in the projects but also for the maximum optimization of resources .Our research stresses on risk oriented testing in pervasive and evolutionary computational areas as due to dynamicity of such computing environment ,the project imbibes risk in great measure and needs to be taken care in early stages of the project .  Risk oriented testing requires concentration in both of these application areas as extremely little work has been done in this regard.


Risk; Risk based testing; Software testing Life cycle; Pervasive computing; Evolutionary Computing

Download this article as: 

Copy the following to cite this article:

Malik V, Singh S. Exploring Computing Environment Possibilities for Risk Oriented Testing. Orient.J. Comp. Sci. and Technol;10(3)

Copy the following to cite this URL:

Malik V, Singh S. Exploring Computing Environment Possibilities for Risk Oriented Testing. Orient. J. Comp. Sci. and Technol;10(3). Available from:


Software testing requires selective and careful planning because it can not be done exhaustively. To increase the test effectiveness proper selection of test cases for testing is mandatory. As we know that risky scenarios requires proper identification because of very limited resources ,lack of time for not complete testing of system we need more focus on the areas where largest risks are found out . For avoiding software failures we need proper identification which can be done by several techniques like by brainstorming, experts knowledge ,brainstorming .

In the present paper we discuss risk oriented or risk based testing application domains and then tries to find out the gaps or shortcomings from the past.

We have gone through various search engines to find out the relevant articles i.e. Springer, IEEE Explore, IEEE computer Society , InderScience ,ACM digital library .

In this paper, we present results of analysis of various application domains of risk oriented testing.

The paper has been structured in various sections to answer the questions as followed:

RQ1: Which application domains for risk based testing has been explored most in the past?

RQ2: Which application areas has not been explored yet in risk oriented testing?

We have explained the risk based testing application areas used in past in section 2 and then have discussed about gaps in section 3 and finally concluded in the section 4.  

Extensive survey has been done on risk oriented testing application areas employed in the past and then we have successfully found out the gaps.

Risk  Based Testing  Application Areas In The Past

Object Oriented Computing Environment

Various object oriented metrics can be used for identifying the classes having high risk values. These metrics can be Number of methods, weighted method per class , response per class ,RFC/NOM ,coupling between objects , depth in tree ,number of children  .Identification of high risk can be done as a class that contains two or more than two metrics which exceed the recommended limits .The purpose is to identify all the high risk classes . Once the risky classes are identified then testing can be done to lessen the risk effect [1].

Cloud Computing Environment

In cloud computing security testing can be applied on 3 layers which are as follows:

  • Service
  • Infrastructure
  • Platform layer

It has be to performed on 2 parties: Cloud providers that offers software as a service and cloud consumers that can develop custom applications .Cloud security can be improved by using risk oriented testing . Negative requirements can be derived by using the risk analysis on cloud under test. Living Models approach can be used which will result in changing risk model by adapting negative requirements. UML can be used for modelling [2] .The following figure 1 depicts the how risk oriented testing can be done in cloud environment.

Figure 1 : Cloud environment risk based testing

Figure 1: Cloud environment risk based testing 

Click here to View figure


Services Based Computing

Various service based computing environment have been used in past for risk based testing. Services may include IT services [3], Web services [4, 5].For IT services risk profiling can be done which is a function of time. By the help of testing risk profiling can be lowered down .Evolution of risk profile can occur with time is shown below in figure 2.

Figure 2 :  Risk profiling over time

Figure 2: Risk profiling over time  

Click here to View figure


In adaptive testing of web services we can assume that the tolerance towards a feature’s failure is basically an inverse ratio to the risk .Risky features are tested earlier than others .Ontology based risk calculation is done for the web services .We require run time monitoring system to find out the dynamic changes in configuration of given services.

Failure probability can be evaluated by first estimating the probability of failure of each ontology class and then by adjusting estimation of each class. Again we find out failure rate at interface services also. We do control flow analysis of whole system and do the final failure probability estimation . We quantitatively analyse risk by 3 layers of web services:

  • Data layer
  • Unit Layer
  • Integration Layer

First complexity based analysis is done which is dependent on how much domain ontology model is complex[4] .All control constructs failure probability is estimated as shown in the figure 3 given below .

 Figure 3: Control Construct Failure Probability

Figure 3: Control Construct Failure Probability 

Click here to View figure


Risk assessment can be done in parallel with test development and we need to decide tactics for service ranking [4]. In the following figure 4 we can see how adaptive testing in groups can be done for web based services.

Figure 4: Adaptive testing of web based services

Figure 4: Adaptive testing of web based services 

Click here to View figure


Web services can also be tested by help of fault tree diagrams in which leaf nodes are used to represent the dangerous scenarios [5] .Here we use state model diagram to look for system activities in order to test web transactions as shown in the figure 5 given below.

Figure 5: Activity state diagram of web transactions based system

Figure 5: Activity state diagram of web transactions based system  

Click here to View figure


While drawing system fault tree we need to consider several system properties i.e. Composition, Sorting, visibility, durability, consistency, recovery, controllability and accordingly can draw fault tree diagram  which is shown below in figure 6 .

Figure 6: Fault tree representation

Figure 6: Fault tree representation 

Click here to View figure


Software development and manufacturing computing environment

Half of the development time can be taken while manufacturing big machines. Proper test sequencing is to be done which can be done by AND OR graphical method. If the test time and cost exceeds the risk cost then we should stop the testing. For a software test phase its possible to evaluate the optimal test sequence which can do risk reduction in a given time frame [6]. The required amount of subsystem testing can be calculated once the reliability tests can be calculated. Geometric distributed path lengths and Bernoulli type inspection errors can be used for reducing risk in software [7]. Hypothesis testing can be done for Type 1 and Type 2 errors which are shown as in the table 1 given as follows:

Table 1 :  Hypothesis testing Type 1 and Type 2 errors

Table 1: Hypothesis testing Type 1 and Type 2 errors 

Click here to View table


Node path representation of the software can also help in finding out defects which can be done as shown in the figure 7 given below :

Figure 7: To find software defects by node path representation

Figure 7: To find software defects by node path representation 

Click here to View figure


Industrial risk oriented testing Computing

Test effectiveness can be improved by considering customer priority and the fault proneness .Prioritization of test cases can be done on these factors. Fault detection rates can be evaluated and based on risk value can be prioritized. Risk based prioritization considers risk exposure as the main metric. Requirements volatile nature and implementation complexity also plays very important role in enterprise risk oriented computing environment .We also consider following metrics for the  further evaluation  which is as follows [8]:

  • Prioritization factor value
  • Weighted priority
  • Total Severity of failures detected

The required database relations can be depicted as follows in the figure 8.

Figure 7 : Main Database relations

Figure 7: Main Database relations  

Click here to View figure


At the enterprise or industry level security concern is also really high. So threat modelling is also done by capturing high risk during the risk analysis phase by designing security test case scenarios. Abnormal behaviour of system is captured by the help of misuse cases [9].  One example of abuse case template is given in the figure 8 given as follows:

Figure 8: Misuse cases example template

Figure 8: Misuse cases example template

Click here to View figure


 If the risk identification and assessment is synthesized successfully then determination of iteration policies, testing order and comprehensive testing can be done effectively.

In the small and medium enterprises which represent 85 % of software industries we require risk oriented testing for continuous process improvement. Mainly risks involved are project risks, business risks and product risks. Only large enterprises have risk complete understanding based on its definition which include probability and impact. More objective oriented process are followed for risk management only for the large enterprises not for the small ones. Informed decisions can be taken if risk information is not vague in nature [10]. Risk based testing can be integrated in the industrial process [11] .Steps for the same is given below: 

  • Plan the Integration Process
  • Identify all risk factors
  • Assess all risky items
  • Design and execute test cases
  • Test planning by risk consideration
  • Continuous improvement

Steps for the same are  shown in the figure 9 given below:

Figure 9 : Integrating risk oriented testing in industrial test process

Figure 9: Integrating risk oriented testing in industrial test process 

Click here to View figure


Risk concept is dependent on context and scope in industries. Even if testing does not incorporate risk explicitly still it plays vital role in implicit manner also. To design the new test cases and for further classification understanding prioritization is also. Information about risk can be used for extension of  scope of testing for the risky areas as here most of the critical issues are located[12] .

Open Source software risk oriented testing Computing

To handle quality risks of open source software first data pre-processing takes place by community detection then monitoring of community takes place out of which anomalies are detected[13]. The figure 10 given below indicates how risk oriented testing takes place in open source software.

Figure 10: Open Source software risk oriented testing

Figure 10: Open Source software risk oriented testing 


Click here to View figure


The community dynamics can be monitored by detection of anomaly between the members of community. Social network of communication is created which contains following as nodes:

  • Team Member
  • Developer
  • User

How can be detect communities is shown below in the figure 11 given below:

Figure 11 : How communities can be detected

Figure 11: How communities can be detected  

Click here to View figure


Based on data processing by community detection we can further apply risk oriented testing as shown in the figure 12 given below :

Figure 12 : Risk oriented Testing by test case selection and prioritization

Figure 12: Risk oriented Testing by test case selection and prioritization

Click here to View figure


Similarity based risk measurement has been applied in past where the execution traces if similar to past failing test cases

then it means degree of  risk is really high in the project . The execution traces can be mined by Daikon tool[14].

Embedded Software Computing and Risk Oriented Testing

Fault tree analysis can be done on embedded systems and accordingly test cases can be derived, selected and prioritized as per severity level identified. Safety analysis can be done by fault tree analysis in which we  have a set of events which are subset of a cut set of the fault tree[15] .WE have the following type of test process in such cases as shown in the  figure 13 given below :

Figure 13: Risk based test process

Figure 13: Risk based test process


Click here to View figure


Networks based risk oriented computing

Various models of system can be utilized for composition of system models .Safety and availability information can be used to model the artifacts of system. Networked system is made of the various components that may have different or common object of control[16]. Operating modes of the system can be of several types i.e. Initializing, clean up ,emergency ,deadhead .Following in the figure 14 is given how fault tree can be constructed and test cases can be generated .

Figure 14 : Test case generation by fault tree analysis

Figure 14: Test case generation by fault tree analysis 

Click here to View figure


Quality oriented Risk based testing

Active continuous quality control can be implemented by automata learning based modelling for prioritization of critical concepts. Alphabetic models are used to steer quality control process to increase the risk coverage. For risk minimization we need to maintain test models, instrumentation, execution environment and test evaluation procedures[17] .Modelling layers for automatic quality control are given as follows  in the figure 15  :

Figure 15 : Quality control modelling layers

Figure 15 : Quality control modelling layers 

Click here to View figure


Quality evaluation based on quality model i.e. QuaMoCo  can be used with risk oriented testing [18].This model also operated on ISO 25010 by providing tool support  for quality assessment .Product quality modelling [19] by ISO 25010 standard has been depicted following in the figure 16 as shown below .

Figure 16 : ISO 25010 quality model

Figure 16 : ISO 25010 quality model


Click here to View figure


Discussion and Findings

Risk oriented testing  has been explored in several areas but no work has been done in case of pervasive computing and very little concentration has been done on evolutionary algorithms.

Pervasive Computing Environment

Also named as ubiquitous computing, pervasive computing environment is highly dynamic and evolutionary in nature .Risk oriented testing has not been applied yet in such type of environment.  Such kind of computing is really complex in nature as we should be aware of user intent, high level energy for management, proactively handling and transparency is also required[20]. We can combine distributed computing and mobile computing to form pervasive computing concepts which should be taken care while considering system risk levels for testing as shown below in figure 16 .

Figure 16: Pervasive computing Issues

Figure 16: Pervasive computing Issues 

Click here to View figure


Risks should be explored based on challenges in computing environment that include heterogeneity in environment, context awareness, access control, privacy , mobility, data communication and trust[21].

Evolutionary Algorithmic computing Environment

Evolutionary computation that is required for automatic design, problems optimization and machine learning and where selection of individual takes place by evaluation of fitness function. Various risks factors include for testing is finding optimal solution ,computationally expensive and requirement for parameter tuning .Evolutionary computation has also not been explored in risk oriented testing due to very dynamic and evolutionary nature of problems .

Conclusions and Future Work

Risk oriented testing computation has been explored in Object oriented, cloud computing, service based computing , software development , manufacturing process ,industrial ,open source software ,model based systems , embedded systems , network based systems and quality oriented systems but no work has been done on pervasive computing and evolutionary computation .So we propose work on both of these computational paradigms and risk oriented testing is required in early stages of the system for proper risk management of the system .


  1. L.Rosenberg, R. Stapko, A. Gallo, “Risk-based object oriented Testing” ,Proceeding of the 24th Annual Software Engineering Workshop, pp. 1–6. NASA, Software Engineering Laboratory,1999
  2. P.Zech, “Risk-based security testing in cloud computing environments , Proceeding of the Fourth International Conference on Software Testing, Verification and Validation (ICST’11), pp. 411–414. IEEE, New York ,2011
  3. J.Sauve, “Risk based Service Testing , Several approaches to the application of risk to IT Service Management”, IEEE Xplore ,106-109 ,2008
  4. X.Bai, R.S Kenett, W. Yu, “Risk assessment and adaptive group testing of semantic web services”,Int. J. Softw. Eng. Knowl. Eng. 22(05), 595–620 ,2012
  5. R. Casado, J. Tuya, M. Younas , “Testing long-lived web services transactions using a risk-based approach” , Proceeding of the 10th International Conference on Quality Software (QSIC’10),pp. 337–340. IEEE, New York ,2010
  6. R Boumen , “Risk-Based Stopping Criteria for Test Sequencing, IEEE Transactions on Systems, Man and Cybernetics”, Part A : Systems and Humans, Vol. 38, 1363-1373  ,2008
  7. T.Bagchi , “Models for software defects and testing strategies” ,SIGSOFT engineering notes , volume 34, pp. 1-4, 2009
  8. H. Srikanth, “  Requirements based test prioritization using risk factors: An industrial study, Information and Software Technology”, 69 ,71–83,2016
  9. K.Murthy, K.RThakkar., S.Laxminarayan, “Leveraging risk based testing in enterprise systems security validation”, Proceeding of the First International Conference on Emerging Network Intelligence (EMERGING’09), pp. 111–116. IEEE,New York ,2009
  10. M. Felderer, “ Risk orientation in software testing processes of small and medium enterprises: an exploratory and comparative study, International Journal of Software Quality J ,24:519–548 ,2016
  11. M.Felderer, R. Ramler , “Integrating risk-based testing in industrial test processes”, Springer Science+ Business Media, Software Qual J ,22:543–575 ,2014
  12. M. Felderer, “ A multiple case study on risk based testing in industry “, International Journal of  software tools technology transfer ,16:609-625,2016
  13. I. Yahav , “Risk Based Testing of Open Source Software”, IEEE 38th Annual International Computers, Software and Applications Conference Workshops, 638-643,2014
  14. T. Noor, “Test Case Analytics: Mining Test Case Traces to Improve Risk-Driven Testing”, International Journal on Software Quality Assurance , IEEE Transactions , Volume-9, Issue-3, Pages 119- 127,2015
  15. J. Kloos , “ Risk based testing of safety critical embedded systems driven by fault tree analysis”, IEEE,2011
  16. T. Hussain, “ Automated fault tree generation and risk based testing of networked automation systems” , IEEE,2010
  17. J. Neubauer , “Risk based testing via active continuous quality control”, International journal of software tools technology transfer , Springer , 2014
  18. H. Foidl , “Integrating software quality models into risk-based testing”, Software Qual J,,2016
  19. ISTQB: Standard glossary of terms used in software testing. version 2.2. Tech. rep., ISTQB ,2012
  20. M. Satyanarayanan, “Pervasive Computing: Vision and Challenges”, IEEE Personal Communications, 2001
  21. L.Bhaskar , “ Pervasive Computing issues, applications and challenges”,International Journal of Engineering and computer science , 2013 
Share Knowledge: Share on LinkedInShare on FacebookTweet about this on TwitterShare on Google+Share on RedditEmail this to someone

Comments are closed.