Tez No	İndirme	Tez Künye	Durumu
180817		A pre-enactment model for measuring process quality / Uygulama öncesi süreç kalitesinin ölçülmesi için bir model Yazar:SELÇUK GÜCEĞLİOĞLU Danışman: DOÇ. DR. ONUR DEMİRÖRS Yer Bilgisi: Orta Doğu Teknik Üniversitesi / Enformatik Enstitüsü / Bilişim Sistemleri Ana Bilim Dalı Konu:Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve Kontrol = Computer Engineering and Computer Science and Control Dizin:	Onaylandı Doktora İngilizce 2006 176 s.
ÖZUYGULAMA ÖNCESİ SÜREÇ KALİTESİNİNÖLÇÜLMESİ İÇİN BİR MODELGüceğlioğlu, A.SelçukDoktora, Bilişim Sistemleri BölümüTez Yöneticisi: Doç.Dr. Onur DemirörsHaziran 2006, 160 sayfaSüreç ölçüm çalışmalarının çoğu zaman ve maliyet tabanlı modeller ile ilişkilidir.Kalite diğer bir klasik görünüş olmasına rağmen, literatürde yaygın olarak kullanılansüreç kalitesini ölçen modeller bulunmamaktadır. Tez kapsamında, kalite hakkındatamamlayıcı bilgiler sağlamak amacıyla bir süreç kalite ölçüm modeliningeliştirilmesi seçilmiş ve süreç karakteristikleri ile ilgili çalışmalar araştırılmıştır.Buna ek olarak, süreç ile yazılım arasındaki benzerliklerden yararlanarak, yazılımkalite çalışmaları incelenmiştir. Yapılan araştırmalar ışığında, ISO/IEC 9126 YazılımÜrün Kalite Modeli temelinde bir model geliştirilmiştir. Model içerisinde, bazı kaliteöznitelikleri süreç karekteristiklerine göre yeniden tanımlanmıştır. Ayrıca, yalnızcasürece özgü yeni özniteliler geliştirilmiştir. Bir durum çalışması gerçekleştirilmiş vesonuçları uygulanabilirlik, anlaşılabilirlik ve uygunluk farklı bakış açılarına göretartışılmıştır.Anahtar kelimeler: Süreç kalitesi, kalite öznitelikleri, bilgi sistemleri etkisi, yazılımkalite karakteristikleri.v
DirectorI certify that this thesis satisfies all the requirements as a thesis for the degree ofDoctor of Philosophy.Assoc.Prof.Dr. Yasemin YARDIMCIHead of the DepartmentThis is to certify that we have read this thesis and that in our opinion it is fullyadequate, in scope and quality, as a thesis for the degree of Doctor of Philosophy.Assoc.Prof.Dr. Onur DEMİRÖRSSupervisorExamining Committee MembersProf. Dr. Semih BİLGEN (METU, EEE)Assoc. Prof. Dr. Onur DEMİRÖRS (METU, IS)Assoc. Prof. Dr. Zeynep ONAY (METU, BA)Prof. Dr. Ünal YARIMAĞAN (ÖSYM)Dr. Altan KOÇYİĞİT (METU, IS)I hereby declare that all information in this document has been obtained andpresented in accordance with academic rules and ethical conduct. I also declarethat, as required by these rules and conduct, I have fully cited and referencedall material and results that are not original to this wok.Name and Surname: A.Selçuk GüceğlioğluSignature:iiiABSTRACTA PRE-ENACTMENT MODEL FOR MEASURINGPROCESS QUALITYGüceğlioğlu, A.SelçukPh.D., Department of Information SystemsSupervisor: Assoc. Prof. Dr. Onur DemirörsJune 2006, 160 pagesMost of the process measurement studies are related with time and cost basedmodels. Although quality is the other conventional aspect, there are no widely usedmodels for measuring the process quality in the literature. In order to providecomplementary information about the quality, a process quality measurement modelhas been chosen to be developed and the studies about process characteristics havebeen searched in the scope of the thesis. Moreover, by utilizing the similaritiesbetween process and software, the studies in software quality have been investigated.In the light of the researches, a model is built on the basis of ISO/IEC 9126 SoftwareProduct Quality Model. Some of the quality attributes are redefined in the modelaccording to the process characteristics. In addition, new attributes unique only to theprocess are developed. A case study is performed and its results discussed fromdifferent perspectives of applicability, understandability and suitability.Keywords: Process quality, quality attributes, Information Systems (IS) effects,software quality characteristic.ivÖZUYGULAMA ÖNCESİ SÜREÇ KALİTESİNİNÖLÇÜLMESİ İÇİN BİR MODELGüceğlioğlu, A.SelçukDoktora, Bilişim Sistemleri BölümüTez Yöneticisi: Doç.Dr. Onur DemirörsHaziran 2006, 160 sayfaSüreç ölçüm çalışmalarının çoğu zaman ve maliyet tabanlı modeller ile ilişkilidir.Kalite diğer bir klasik görünüş olmasına rağmen, literatürde yaygın olarak kullanılansüreç kalitesini ölçen modeller bulunmamaktadır. Tez kapsamında, kalite hakkındatamamlayıcı bilgiler sağlamak amacıyla bir süreç kalite ölçüm modeliningeliştirilmesi seçilmiş ve süreç karakteristikleri ile ilgili çalışmalar araştırılmıştır.Buna ek olarak, süreç ile yazılım arasındaki benzerliklerden yararlanarak, yazılımkalite çalışmaları incelenmiştir. Yapılan araştırmalar ışığında, ISO/IEC 9126 YazılımÜrün Kalite Modeli temelinde bir model geliştirilmiştir. Model içerisinde, bazı kaliteöznitelikleri süreç karekteristiklerine göre yeniden tanımlanmıştır. Ayrıca, yalnızcasürece özgü yeni özniteliler geliştirilmiştir. Bir durum çalışması gerçekleştirilmiş vesonuçları uygulanabilirlik, anlaşılabilirlik ve uygunluk farklı bakış açılarına göretartışılmıştır.Anahtar kelimeler: Süreç kalitesi, kalite öznitelikleri, bilgi sistemleri etkisi, yazılımkalite karakteristikleri.vTo my son, YAĞIZviACKNOWLEDGEMENTSI would like to express my most sincere appreciation to my supervisor Assoc. Prof.Dr. Onur DEMİRÖRS for his continuous support and intellectual guidancethroughout this work and his very valuable friendship.I would like to thank to Prof. Dr. Semih BİLGEN and Assoc. Prof. Dr. ZeynepONAY for their critical remarks and valuable guidance during the whole period ofmy research.I would like to acknowledge the participants in the case study who spent their timeand shared their views and experience with me.I would like to thank to Yelda KUŞCU for her review and valuable feedback.I would like to thank to my wife, ?Yeşim,? and my son, ?Yağız,? my mother, myfather for their continuous patience, understanding and emotional support duringthese years.viiTABLE OF CONTENTSSELF DECLARATION AGAINST PLAGIARISM………………………..…....iiiABSTRACT………………………………………………………………………....ivÖZ……………………………………………………………………………….……vDEDICATION...……………………………………………………………………viACKNOWLEDGEMENTS……….……………………………………….……..viiTABLE OF CONTENTS…………………………………………………………viiiLIST OF TABLES..………………………………………………………………..xiiLIST OF FIGURES…..……………………………………………………………xvCHAPTER1 INTRODUCTION.............................................................................................. 11.1 THE NEED FOR A PROCESS QUALITY MEASUREMENT ................................... 11.2 PROPOSING A MODEL FOR MEASURING THE PROCESS QUALITY ................... 31.3 METHOD EMPLOYED FOR VALIDATION OF THE MODEL ................................ 51.4 THESIS OUTLINE ........................................................................................... 62 LITERATURE REVIEW.................................................................................. 82.1 EFFECTS OF INFORMATION SYSTEM (IS) STUDIES ON THE PROCESS ............. 82.1.1 DeLone and McLean IS Success Model .............................................. 82.1.2 The Updated DeLone and McLean IS Success Model ...................... 122.1.3 Seddon IS Effectiveness Matrix......................................................... 142.1.4 A Contingency Theory for IS Assessment......................................... 162.1.5 Process-Based Approaches for Measuring IS Effects........................ 192.1.5.1 Mooney?s Process Oriented Framework........................................ 192.1.5.2 A Process-Based Model for the Organizational Impact of IS........ 212.1.5.3 Özkan?s Framework for the Assessment of IS Effectiveness ........ 26viii2.2 ANALYSIS AND IMPROVEMENT OF THE PROCESSES ..................................... 262.2.1 Characteristics of the Process............................................................. 272.2.2 Business Process Change (BPC) Studies ........................................... 282.2.3 Techniques Employed in BPC Studies .............................................. 292.2.4 The Relations between BPC Studies and IS Development................ 302.2.5 Categorization of IS Effects on the Process....................................... 313 A NEW MODEL FOR MEASURING PROCESS QUALITY.................... 333.1 MODEL OBJECTIVES.................................................................................... 333.2 MODEL FOUNDATIONS ................................................................................ 343.3 MODEL SCOPE AND DETAILS ...................................................................... 363.4 MODEL USAGE............................................................................................ 363.5 QUALITY CATEGORY .................................................................................. 393.5.1 Maintainability Metrics...................................................................... 393.5.1.1 Analyzability Metrics..................................................................... 403.5.2 Reliability Metrics.............................................................................. 433.5.2.1 Fault Tolerance Metrics ................................................................. 443.5.2.2 Recoverability Metrics ................................................................... 443.5.3 Functionality ...................................................................................... 483.5.3.1 Suitability Metrics.......................................................................... 483.5.3.2 IT Based Functionality Metrics...................................................... 493.5.3.3 Accuracy Metrics ........................................................................... 513.5.3.4 Interoperability Metrics.................................................................. 533.5.3.5 Security Metrics ............................................................................. 553.5.4 Usability ............................................................................................. 563.5.4.1 Understandability Metrics.............................................................. 563.5.4.2 Learnability Metrics ....................................................................... 563.5.4.3 Operability Metrics ........................................................................ 573.5.4.4 Attractiveness Metrics.................................................................... 604 A CASE STUDY FOR MEASURING PROCESS QUALITY .................... 624.1 CASE STUDY RESEARCH IN INFORMATION SYSTEMS .................................. 624.2 PURPOSE OF THE CASE STUDY .................................................................... 634.3 CASE STUDY DESIGN .................................................................................. 64ix4.3.1 Components of the Case Study Design.............................................. 644.3.1.1 Research Questions ........................................................................ 644.3.1.2 Case Definition .............................................................................. 664.3.1.3 Interpreting the Findings ................................................................ 684.4 CONDUCTING CASE STUDY ......................................................................... 684.4.1 Collecting the Data............................................................................. 684.4.2 Arranging the Data into Documents .................................................. 724.4.3 Applying the Model to the Processes................................................. 734.4.3.1 Sample Process, Material Request (AS-IS Form).......................... 754.4.3.1.1 Details in the Process Definition Document ............................ 754.4.3.1.2 Details of the Problems in the Processes Document................ 824.4.3.1.3 Details in the Quality Measurement Document ....................... 834.4.3.2 Sample Process, Material Request (TO-BE Form) ........................ 884.4.3.2.1 Details of the Process Definition Document............................ 884.4.3.2.2 Details of the Problems in the Processes Document................ 944.4.3.2.3 Details in the Quality Measurement Document ....................... 944.4.4 Effort Spent in the Case Study ........................................................... 984.5 ANALYZING THE CASE STUDY MEASUREMENTS......................................... 994.5.1 Examining Quality Attribute Values................................................ 1004.5.1.1 Maintainability Measurements..................................................... 1004.5.1.2 Reliability Measurements............................................................. 1034.5.1.3 Functionality Measurements ........................................................ 1084.5.1.4 Usability Measurements............................................................... 1184.5.2 Relations among the Problems and Quality Attributes .................... 1274.5.2.1 Material Request Process ............................................................. 1274.5.2.2 Meeting Material Request Process............................................... 1294.5.2.3 Material Purchasing Process ........................................................ 1324.5.2.4 Material Registration Process ...................................................... 1344.5.2.5 Material Counting Process ........................................................... 1364.5.2.6 Material Returning Process .......................................................... 1374.5.2.7 Material Record Deletion Process................................................ 1394.5.2.8 Material Repair and Maintenance Process................................... 1414.5.3 Relations Among Quality Attributes................................................ 143x4.5.4 Answering the Research Questions.................................................. 1454.5.5 Closure Part of the Case Study ........................................................ 1475 CONCLUSIONS ............................................................................................ 1505.1 CONTRIBUTION OF THE STUDY.................................................................. 1505.2 LIMITATIONS............................................................................................. 1525.3 FUTURE RESEARCH ................................................................................... 153REFERENCES……………………………………………………………………155VITA………..….………………………………………………………………….159xiLIST OF TABLESTable 2.1 Three Studies: Shannon and Weaver, Mason and DeLone and McLean………………………………………………………………………...………………9Table 2.2 IS Effectiveness Measures for Different Combinations of System andStakeholder…………………………………………………………….……………15Table 2.3 Results of the IS Effects on Business Process Attributes………………..20Table 3.1 Metric Details…………………………………………………………….38Table 3.2 Complexity Metric……………………………………………………….42Table 3.3 Coupling Metric………………………………………………………….43Table 3.4 Failure Avoidance Metric………………………………………………..45Table 3.5 Restorability Metric……………………………………………………...46Table 3.6 Restoration Effectiveness Metric………………………………………...47Table 3.7 Functional Adequacy Metric……………………………………………..49Table 3.8 Functional Completeness Metric…………………………………………50Table 3.9 IT Usage Metric………………………………………………………….51Table 3.10 IT Density Metric……………………………………………………….52Table 3.11 Computational Accuracy Metric………………………………………..53Table 3.12 Data Exchangeability Metric……………………………………………54Table 3.13 Access Auditability Metric……………………………………………..55Table 3.14 Functional Understandability Metric…………………………………...57Table 3.15 Existence in Documents Metric………………………………………...58Table 3.16 Input Validity Checking Metric………………………………………...59Table 3.17 Undoability Metric……………………………………………………...60Table 3.18 Attractive Interaction Metric……………………………………………61Table 4.1 Detailed Process Definitions within Activity Level……………………...69Table 4.2 Total Number of Activity for each AS-IS Form Process.………………..69xiiTable 4.3 Forms, Tools, Documents Employed in the AS-IS Form Processes……..70Table 4.4 Total Number of Activity for each TO-BE Form Process……………….71Table 4.5 Forms, Tools, Documents Employed in the TO-BE Form Processes……72Table 4.6 Maintainability Attributes Measurement………………………………...73Table 4.7 Reliability Attributes Measurement……………………………………...74Table 4.8 Functionality Attributes Measurement…………………………………..74Table 4.9 Usability Attributes Measurement…………………………………….…74Table 4.10 Material Request Activities (AS-IS)……………………………………76Table 4.11 Maintainability Measurement of Material Request Process (AS-IS)…...84Table 4.12 Reliability Measurement of Material Request Process (AS-IS)………..84Table 4.13 Functionality Measurement of Material Request Process-I (AS-IS)……………………………………………………………………………………85Table 4.14 Functionality Measurement of Material Request Process-II (AS-IS)……………………………………………………………………………………86Table 4.15 Usability Measurement of Material Request Process (AS-IS)……….…87Table 4.16 Measurement Summary for Material Request Process (AS-IS)………..89Table 4.17 Material Request Activities (TO-BE)…………………………………..90Table 4.18 Maintainability Measurement of Material Request Process (TO-BE)….95Table 4.19 Reliability Measurement of Material Request Process (TO-BE)…….…95Table 4.20 Functionality Measurement of Material Request Process-I (TO-BE)…..95Table 4.21 Functionality Measurement of Material Request Process-II (TO-BE)…96Table 4.22 Usability Measurement of Material Request Process (TO-BE)………...96Table 4.23 Measurement Summary for Material Request Process (TO-BE)……….97Table 4.24 Efforts Spent in the Case Study………………………………………...98Table 4.25 Measurement Values of Complexity Attribute..…..…………………..101Table 4.26 Measurement Values of Coupling Attribute…..…..…………………..102Table 4.27 Measurement Values of Failure Avoidance Attribute…..…..………...104Table 4.28 Measurement Values of Restorability Attribute…….…..…..………...106Table 4.29 Measurement Values of Restoration Effectiveness Attribute…..…..…107Table 4.30 Measurement Values of Functional Adequacy Attribute…..…..…..….109Table 4.31 Measurement Values of Functional Completeness Attribute…..…..….110xiiiTable 4.32 Measurement Values of IT Usage Attribute……………….…..…...…111Table 4.33 Measurement Values of IT Density Attribute…………………..…..…112Table 4.34 Measurement Values of Computational Accuracy Attribute.…..…..…114Table 4.35 Measurement Values of Data Exchangeability Attribute…..…..…..…115Table 4.36 Measurement Values of Access Auditability Attribute………..…..….117Table 4.37 Measurement Values of Functional Understandability Attribute……..118Table 4.38 Measurement Values of Existence in Documents Attribute………..…120Table 4.39 Measurement Values of Input Validity Checking Attribute………..…122Table 4.40 Measurement Values of Undoability Attribute……….………..…..….124Table 4.41 Measurement Values of Attractive Interaction Attribute………..…….125xivLIST OF FIGURESFigure 1.1 Model Structure….……………………………………………………….5Figure 2.1 DeLone and McLean IS Success Model………………………………...10Figure 2.2 Updated DeLone and McLean IS Success Model………………………13Figure 2.3 IS Assessment Model…………………………………………………...18Figure 2.4 IS Effects on Business Process Attributes……………………………....21Figure 2.5 Soh and Markus Model with Three Processes…………………………..22Figure 2.6 A Process-Based Model for the Organizational Impact of IS…………..23Figure 2.7 Research Model: Investigating the Effects of IS on OrganizationalPerformance…………………………………………………………………………24Figure 3.1 Specified Attributes.…………………………………………………….39Figure 4.1 Supply Chain Department and its Sections……………………………..67Figure 4.2 Material Request Activities-I (AS-IS)…………………………………..80Figure 4.3 Material Request Activities-II (AS-IS)………………………………….81Figure 4.4 Material Request Activities-III (AS-IS)………………………………...82Figure 4.5 Measurement of Material Request Process (AS-IS Form)……………...89Figure 4.6 Material Request Activities (TO-BE)…………………………………...93Figure 4.7 Measurement of Material Request Process with AS-IS and TO-BE Values………………………………………………………………….……………………97Figure 4.8 Complexity Values of AS-IS and TO-BE Processes…………………..102Figure 4.9 Coupling Values of AS-IS and TO-BE Processes……………………..103Figure 4.10 Failure Avoidance Values of AS-IS and TO-BE Processes………….105Figure 4.11 Restorability Values of AS-IS and TO-BE Processes………………..106Figure 4.12 Restoration Effectiveness Values of AS-IS and TO-BE Processes.….108Figure 4.13 Functional Adequacy Values of AS-IS and TO-BE Processes………109Figure 4.14 Functional Completeness Values of AS-IS and TO-BE Processes…..110xvFigure 4.15 IT Usage Values of AS-IS and TO-BE Processes……………………112Figure 4.16 IT Density Values of AS-IS and TO-BE Processes………………….113Figure 4.17 Computational Accuracy Values of AS-IS and TO-BE Processes…..114Figure 4.18 Data Exchangeability Values of AS-IS and TO-BE Processes………116Figure 4.19 Access Auditability Values of AS-IS and TO-BE Processes………...117Figure 4.20 Functional Understandability Values of AS-IS and TO-BEProcesses…………………………………………………………………………...119Figure 4.21 Existence in Documents Values of AS-IS and TO-BE Processes……121Figure 4.22 Input Validity Checking Values of AS-IS and TO-BE Processes……123Figure 4.23 Undoability Values of AS-IS and TO-BE Processes…………………124xviCHAPTER 1INTRODUCTION1 IntroductionThis chapter is divided into four sections. The first section explains the need for aprocess quality measurement. The second section presents the model developed inthe thesis for measuring the process quality. The third section describes the methodemployed for validating the model. Finally, the last section gives outline of thethesis.1.1 The Need for a Process Quality MeasurementProcess is one of the most valuable assets of the organizations. Its design andimplementation have considerable impacts on the success of an organization. For thisreason, process-centered studies are emphasized in the literature as a necessity foraccomplishing business goals (Davenport, 1993; Hammer 2001).Information Systems (IS) is an aspect that profoundly affects the process and itsdesign (Mooney, Gurbaxani and Kraemer, 1996). In an IS development project,frequently, processes of an organization are analyzed and a system is designed withnew process definitions. Most of the studies in the IS literature employ time and costbased models and attributes such as productivity growth, return on investment andmarket share for measuring effects of IS projects on the organizations (Brynjolfssonand Hitt, 1994). These models can provide the organizations with crucial informationabout IS effects, but, naturally, they can only be measured during or after theprocesses are executed. In order to indicate this measurement time, these kinds ofattributes are named as post-execute attributes in the thesis. The processes should bemodified according to post-execute attribute values and re-executed to measure theeffects of new arrangements. This kind of iterations requires much effort and cost.1There are also other problems with using post-execute attributes. The first problem isto have the difficulties in identifying IS effects. Available models do not identify ISeffects on the process. What kind of changes in the process affect the measurementsis usually not a primary issue. The second problem is related with the difficulties inisolating contributions of IS effects from other contributors. As there are more thanone factor affecting the process, it is difficult to isolate and measure only IS effects.And finally economic performance might not be the most critical factor for someorganizations such as public organizations. As the economic criteria are not someaningful for these organizations, only productivity gains can be used to measureIS effects on such organizations (Danziger, 1987). These problems are also stated inone of the most well-known IS Success Model (DeLone and McLean, 1992). In themodel, available studies are accepted as in early stages and much work is suggestedto be performed for measuring IS effects on the organizations.In addition to time and cost related attributes, another important aspect for theprocess is quality. However, there is limited number of attributes defined in thequality aspect. Complexity and dependency are examples of such attributes.Complexity is related to comprehensibility of a process (Hammer and Steven, 1994).Since the comprehensibility of the process, with its inputs, activities and outputs, byeach member of the organization is vital for the effectiveness, it makes complexity aprecious attribute for the process. Dependency, which is mentioned as the secondexample, focuses on the interactions with other processes in the organization(Hammer, 1996). Higher number of interactions with other processes increases thepossibility of a delay in the process because of higher time consumption whilegetting responses. In such a case, dependency attribute may help identifying thereasons of the delays by focusing on the interactions in order to minimize thesedelays. Measuring these attributes by using process definitions can provide earlyfeedback and reveal critical problems before the processes are executed. These kindsof attributes are called as pre-execute attributes in the thesis to indicate measurementtime. Unfortunately, there are no widely accepted and used models including pre-execute attributes for measuring the process quality in different perspectives such asusability, reliability and maintainability.2Business Process Change (BPC), whether in the form of improvement orreengineering, is another study affecting the process. In a BPC study, organizationsanalyze their processes, address the barriers that impede their ability and identifynew ways to improve them. In this perspective, BPC has close relationships with ISdevelopment projects. BPC studies allow IS analysts to capture what a business doesby answering some important questions such as ?where bottlenecks are,? ?whereadditional resources are needed,? ?where commitments are not being satisfied? and?how processes can be optimized.? When implemented in coordination the answersof the questions asked during the BPC initiative form a basis for IS development.Therefore, not only IS projects have close relationship with the BPC studies but alsothe quality of them will directly affect the quality of the IS employed. Based on BPC,various techniques such as Activity Based Costing (ABC) and Cost Benefit Analysis(CBA) are proposed for measuring the processes. These models use time and costrelated post-execute attributes such as cycle time, throughput time, cost reductionsand revenue increases for measuring the impacts of the studies on the processes,similar to the models employed for measuring IS effects on the process. In thiscircumstance, there is a need to measure the impacts of the BPC studies on theprocess quality with using pre-execute attributes for having early feedback.1.2 Proposing a Model for Measuring the Process QualityThe literature research has demonstrated the lack of frameworks for defining andmeasuring process quality. To enable quantitative evaluation of quality attributes, aprocess quality measurement model is developed in this study. At the beginning ofthe development, Goal Question Metric (GQM) method (Basili, 1992) was used toinvestigate quality attributes. Some attributes such as complexity, dependency,consistency, information technology (IT) usage and interoperability were defined.However, these findings remained to be organization specific and it was difficult togeneralize these metrics to form a more comprehensive model.During the literature survey, the analogy between software and process (Osterweil,1987) has given a useful direction to the study. Process and software have similarlogical structures. For instance, structure of the process with its inputs, activities andoutputs is similar to that of the software with its input parameters, functions and3output parameters. The relation between software and function exists betweenprocess and activity. Software and process constitutes a part of the whole and hasinteractions with other parts as well. In addition, high quality is of prime importancefor both of them.In order to provide a more complete quality attribute set, available software qualitymodels were examined (Boehm, Brown and Lipow, 1976; McCall, Richards andWalters, 1977; ISO/IEC FCD 9126-1.2, 2000). ISO/IEC 9126 Software ProductQuality Model, one of the most widely accepted models, was used to extend thestudy (ISO/IEC FCD 9126-1.2, 2000). The standard describes a software evaluationmodel for developing or selecting high quality software. The software product isevaluated for every relevant quality characteristics in the model by using validatedand widely accepted metrics. Not only process characteristics that are determinedduring the literature review but also similarities between process and software areused for developing the model. Process quality attributes are redefined according tothe process characteristics; and new attributes unique to the process are developed toextend the model.The model is designed in four-leveled structure that is similar to the ISO/IEC 9126,as depicted in Figure 1.1. The first level is called as category. There is one categoryas ?quality? but in the future, new categories can be added to extend the model. Thesecond level is called as characteristic. The quality category includes Maintainability,Functionality, Reliability and Usability characteristics. The third level is for sub-characteristics and finally, fourth level is for metrics. There are 17 metric definitionsin the model to measure the quality attributes. Similar to the development of newcategories, new characteristics, sub-characteristics and metrics can be defined in themodel as well.The first objective of the model is to measure the quality attributes by using processdefinitions with their inputs, activities and outputs; therefore, providing earlyfeedback about the processes before they are put into execution. The qualitymeasurement results can also be used with post-execute attributes as complementarypurpose. The second objective is to give information about effects of the studies such4as BPC and IS on the process, in terms of quality attributes. The changes in thequality attribute values can indicate effects of the study on the process. In this way,new studies can be organized to increase or decrease specific quality attribute valuesaccording to the priorities of the study..........Quality New CategoriesNewMaintainability Reliability Functionality Usability ....... CharacteristicsNewAnalyzability ......... Sub-CharacteristicsNewComplexity Coupling ......... AttributesFigure 1.1 Model Structure1.3 Method Employed for Validation of the ModelCase study research method was used for investigating applicability, usability andsuitability of the quality attributes, as well as, validating and evolving the model.Before the case study, a pilot study was performed on a single process (Demirors andGuceglioglu, 2005). In the pilot study, the model was applied to AS-IS and TO-BEforms of the process. TO-BE form of the process was designed according to thespecifications of an IS project. The quality attribute values were measured for bothforms. This pilot study provided the following valuable feedback to the modeldevelopment.• The usage of process definitions to measure process quality was experimentedand quality attributes were measured for both forms.• The changes from AS-IS form to the TO-BE form were measured in terms ofquality attributes? values. In addition, IS effects on the process quality were5measured by means of restoration effectiveness, IT usage and IT densityattributes.• Quality attribute definitions were detailed by adding new fields such as ?input tomeasurement,? ?focus,? ?assumptions and constraints? and ?questions foridentifying the attributes in activity definition? to make the measurement clearer.The results of the pilot study provided motivation to design and conduct a case study.After completing the modifications on the model, a case study was designed for anorganization where the pilot study was performed. The case study design wasdescribed with research questions, data collection and data analysis methods. Similarto the pilot study, the model was applied to both AS-IS and TO-BE forms of theprocesses. The quality attribute values of both forms of the processes were measuredby using process definitions.The problems or difficulties encountered in the AS-IS form processes were recordedfrom the documents prepared in the analysis of processes. These problems and theirpresent situations in the TO-BE form were compared with the changes in the qualityattribute values from AS-IS to TO-BE. It is recognized that the improvements in thequality attribute values could indicate the solutions provided to the problems in theTO-BE form processes. The connections among the problems and quality attributeswere used to validate the model. The measurement values of the interrelated qualityattributes provided consistent results for the validation as well.The case study results were discussed with the participators at the closure part of thestudy. The applicability, usability and suitability of the model were evaluated bymeans of mutual open-ended questions. The answers provided another means for thevalidation. The comments stated by the participators and measurement values gavemeaningful answers to the research questions determined at the beginning of the casestudy.1.4 Thesis OutlineThe remainder of the thesis is organized as follows. Chapter 2 presents a literaturereview for IS and BPC effectiveness measurements. Firstly, IS success and its6measurement is given. The most well known IS Success Models and their points ofview about IS effects on the process are investigated. Secondly, BPC, the other studyaffecting the process, is evaluated with its objectives, steps and techniques employedfor measuring the processes by using time and cost based models and metrics. Thetight relationships between BPC and IS studies are emphasized in this circumstanceas well. Thirdly, and finally, available sample models and their contributions areexamined. With the available time and cost related metrics, the need for a processquality measurement is identified.Chapter 3 describes development of the model, where model objectives anddefinitions are given. The structure of the model is detailed with its characteristicssub-characteristics and metrics. The quality attribute definitions categorized undermaintainability, reliability, functionality and usability are introduced. The guidelinesfor the usage of each quality attribute are explained.Chapter 4 includes case study research conducted to refine and validate the model. Itdescribes a brief justification for the research method and details about case studydesign with research questions, data collection and analysis methods. The stepsperformed for forming process definitions of AS-IS and TO-BE forms, modelingthem with using a process-modeling tool, identifying and recording problemsencountered in the AS-IS form processes and details of the measurements areexplained. The results are discussed and other experiences acquired during the casestudy are given.Chapter 5 gives a short summary of the study and emphasizes the contributions ofthe model. It further states limitations of the model such as needs of additionalquality attribute definitions and deficiencies of some present quality attributes. Thepropositions for overcoming the limitations and the development of a tool for makingthe measurement easier are given as future study.7CHAPTER 2LITERATURE REVIEW2 Literature ReviewThis chapter provides a review of the related literature and is divided into twosections. The first section investigates the effects of Information System (IS) studieson the process. The most well-known IS Success Models are examined in thiscontext to determine available studies for measuring the effects of IS on the process.The second section is a literature review concerning business process change studies.2.1 Effects of Information System (IS) Studies on the ProcessThe studies in IS literature about measuring the IS effects are brought together andclassified in the studies named as IS Success Models. The most well known modelsare DeLone & McLean IS Success Model (DeLone and McLean, 1992), Seddon ISEffectiveness Matrix (Seddon, Staples, Patnayakuni and Bowtell, 1999) andContingency Theory for IS Assessment (Myers, Kappelman and Prybutok, 1997).These models and their points of view for measuring the effects of IS on the processare investigated below.2.1.1 DeLone and McLean IS Success ModelOne of the most widely known frameworks for measuring the effectiveness of IS isDeLone and McLean IS Success Model (DeLone and McLean, 1992). The purposeof the model is to synthesize previous researches including IS success into a morecoherent body of knowledge and to provide guidance to future researchers. In themodel, DeLone and McLean (1992) attempted to combine individual measuressystematically from the IS success categories to create a comprehensivemeasurement instrument. This model is based on the Shannon and Weavercommunication theory (Shannon and Weaver, 1949) and the studies of Mason(Mason, 1978) on the communication theory.8According to the theory of Shannon and Weaver (1949), information can bemeasured at three different levels as technical level, semantic level and effectivenesslevel. These levels are defined as follow: Technical level is the accuracy andefficiency of the system, which produces the information, semantic level is thesuccess of the information in conveying intended meaning and effectiveness level iseffect of the information on the receiver. The concept of output levels fromcommunication theory demonstrates the ?serial nature? of information. Theinformation flows through a series of stages from its production through its use orconsumption to its influence on individual and/or organizational performance(depicted in the first row of Table 2.1).Table 2.1 Three Studies: Shannon and Weaver (1949), Mason (1978) and DeLoneand McLean (1992)Technical Semantic Effectiveness or Influence LevelShannonLevel Level& Weaver(1949)Production Product Receipt Influence on Influence onMasonReceipt System(1978)System Information Use & Individual OrganizationalDeLone &Quality Quality User Impact ImpactMcLeanSatisfaction(1992)Mason (1978) extended Shannon and Weaver (1949) model by renamingeffectiveness as influence and defined the influence level of information as ?ahierarchy of events which take place at the receiving end of an information systemwhich may be used to identify the various approaches that might be used to measureoutput at the influence level? (depicted in the second row of Table 2.1). This series ofinfluence events includes the receipt of the information, the evaluation of theinformation and the application of the information. Information application leads to achange in recipient behavior and change in system performance. Mason?s adaptationof communication theory to the measurement of IS suggests that there may be a needto be separate success measures for each level of information.9Based on the studies of Shannon and Weaver (1949) and Mason (1978), DeLone andMcLean (1992) developed IS Success Model. They introduced a comprehensivetaxonomy to organize different research studies about defining IS success, as well asto present a more integrated view of IS success concept. This taxonomy has sixmajor dimensions of IS success as System Quality, Information Quality, InformationUse, User Satisfaction, Individual Impact and Organizational Impact (see Figure2.1). In the DeLone and McLean IS Success Model (1992), System Quality measurestechnical success, Information Quality measures semantic success; and Use, UserSatisfaction, Individual impacts and Organizational impacts measure effectivenesssuccess (depicted in the third row of Table 2.1).System UseQualityIndividual OrganizationalImpact ImpactInformation UserQuality SatisfactionFigure 2.1 DeLone and McLean IS Success Model (1992)A total of 180 articles published during 1981-1987 were reviewed and then thesearticles were organized according to the dimensions of the taxonomy. In the SuccessModel, DeLone and McLean (1992) emphasize the definition of IS dependentvariable as necessity. They state that without a well-defined dependent variable,much of IS research is purely speculative.The first dimension, system quality, measures the information system itself and mostof its measures are fairly straightforward, reflecting more engineering-orientedperformance characteristics of the system in question. The second dimension,information quality, focuses on the quality of the information system output, in other10words the quality of the information that the system produces, primarily in thereports forms. Most measures of the information quality are from the perspective ofthe user thus fairly subjective in character. Also these measures are often included asa part of the measures of user satisfaction. The third dimension, information use isabout recipient consumption of the output of an information system. The usage ofinformation system reports, or of management science/operation research models areplaced in this dimension. The fourth dimension, user satisfaction, is defined for therecipient response to the use of the output of an information system. User satisfactionis probably the most widely used single measure of the IS effectiveness for thispurpose. The fifth dimension, individual impact, is related about the effect ofinformation on the behavior of the recipient. It is closely related to performance. Thisimpact is an indication such that an information system has given the user a betterunderstanding of the decision context, improved his or her decision makingproductivity, produced a change in user activity, or changed the decision maker?sperception of the importance or usefulness of the information system.The sixth and the last dimension is organizational impact. This dimension focuses onthe effect of IS on organizational performance. When the available studies in thisdimension are investigated, it is recognized that most of the studies considermeasurement of post-execute economic attributes such as costs, contributions tocompany profits and return on investment (Emery, 1971). Cost Benefit Analysis(CBA) technique is used to measure IS effects on these metrics. However, as theeconomic criteria are not so meaningful for government agencies, Danziger (1987)proposed using productivity gains to measure the IS effects on the organization. Heexplained that productivity gains occur when the functional output of an organizationis increased at the same or increased quality with the same or reduced resourceinputs. He included five productivity measures; staff reduction, cost reduction,increased work volume, new information, and increased effectiveness in publicservices.There are also a few problems for measuring the IS effects on the organizationalimpact dimension. One of them is limited understanding of IS effects. The focus onthe firm level output variables, while important, does not clearly identify IS effects11on the organization and its working (Mooney et al, 1996). The other one is thedifficulty of isolating the effect of the IS effort from other effects which influence theorganizational performance (DeLone and McLean, 1992).In the light of the available studies in the organizational impact dimension, DeLoneand McLean (1992) state that the studies about measuring the IS effects on theorganizations and processes are at the initial stage and much work is required to bedone in categorizing and measuring the changes in the organizations and workpractices, and relating them to the IS.2.1.2 The Updated DeLone and McLean IS Success ModelDue to the progress in terms of the impacts of IS on business and society as well asprogress in IS research during the ten years from the first model, the IS SuccessModel is updated (DeLone and McLean, 2003). In the updated model (see Figure2.2), a new dimension, service quality is added. Service quality is an importantdimension of IS success, especially in the e-commerce environment where customerservice is crucial. It is stated that as commonly used measures of IS effectivenessfocus on the products rather than the service of the IS function, there is a danger thatIS researchers will mismeasure IS effectiveness if they do not include a measure ofIS service quality in their assessment package. The researchers believe that properlymeasured service quality deserves to be added to system quality and informationquality as components of IS success. Although a claim could be made that servicequality is merely a subset of the model?s system quality, the changes in the role of ISover the last decade argue to define it as a separate variable.Another change is the decomposition of Use into Use and Intention to Use. In orderto overcome the difficulties in interpreting the multidimensional aspects of Use, suchas mandatory versus voluntary and informed versus uninformed, DeLone andMcLean (2003) suggest that ?Intention to Use? may be a worthwhile alternativemeasure in some contexts. ?Intention to use? is an attitude, whereas ?use? is abehavior. This substitution may resolve some of the process versus causal concerns.However, attitudes, and their links with behavior, are difficult to measure, and manyresearchers may choose to stay with use with a more informed understanding of it.12InformationQualityIntentionUseto UseNetSystemBenefitsQualityUserSatisfactionServiceQualityFigure 2.2 Updated DeLone and McLean IS Success Model (2003)The last change is combination of individual and organizational impacts into a singlevariable, net benefit. DeLone and McLean (2003) preferred to use net benefitsbecause the impacts of the original term may be positive or negative, thus resulting ina possible confusion as to whether the results are good or bad. Also, no outcome iswholly positive, without any negative consequences. Thus, net benefits are probablythe most accurate descriptor of the final success variable.This new variable, net benefits, raises three issues that must be taken into account:What qualifies as a benefit?, for whom?, and at what level of analysis?. Thedefinition of net benefit will have a broader range than individual and organizationalimpact. As it is stated in the former model, DeLone and McLean (2003) againemphasize in the new model that more field study is necessary to investigate andincorporate net benefit measures.132.1.3 Seddon IS Effectiveness MatrixSeddon IS Effectiveness Matrix, providing a different point of view, is developed formeasuring IS Success (Seddon et al, 1999). Seddon proposed a two-dimensionalmatrix in the model for classifying IS effectiveness measures (depicted in Table 2.2).The first dimension is the type of system studied and includes a range from a singleIT application, a type of IT or IT applications, all IT applications used by anorganization or sub-organization, an aspect of a system development methodology,and to the IT function of an organization or sub-organization. The second dimensionis the stakeholder whose interests the system is being evaluated. There are five typesof stakeholder as an independent observer, an individual user, a group of users, themanagement or owners of the organization and a country or mankind.The matrix was tested to classify IS effectiveness measures from 186 empiricalpapers in three major IS journals during the nine years from 1988 to 1996. In theclassification, IS effectiveness matrix provided a useful guide for conceptualizingeffectiveness measurement in IS research, and for choosing appropriate measures byconcerning the system and stakeholder types.In the Seddon?s IS Effectiveness Matrix study, the authors gave some interpretationsabout DeLone & McLean IS Success Model (1992). They recognize importantcontributions of DeLone and McLean?s Model to the literature on IS successmeasurement, but emphasizes that different stakeholders in an organization mayvalidly come to different conclusions about the success of the same IS. So, accordingto their points of view, it is not sensible to systematically combine the measures fromtheir six IS success categories in measuring IS success, instead, because of the rangeof different systems, stakeholders, and issues involved in different studies, a widediversity of sharply-focused dependent variables is essential. Seddon claims thatdifferent stakeholders and different types of systems require very different measuresof IS effectiveness. He concludes that as IS research covers a multitude of topics, thenotion of ?IS effectiveness? is not an appropriate dependent variable (Seddon et al,1999).14Table 2.2 IS Effectiveness Measures for Different Combinations of System andStakeholderStakeholder/ an aspect of a single IT a type of IT all IT an aspect of a an ITinterest group IT design or application or IT applications system function inuse in an application used by an development anorganization organization organizationmethodologyor suborganizationIndependentobserverIndividualprimaryfocus:Individualbetter-offnessGroupprimaryfocus: Groupbetter-offnessManagementor Ownersprimaryfocus:Organizationalbetter-offnessA Countryprimaryfocus:Society?sbetter-offnessIn the Seddon IS Effectiveness Matrix, similar to the DeLone and McLean ISSuccess Model (1992), post-execute economic measures such as firm growth, costsavings, return on assets, percentage change in labor, and market share are used formeasuring the IS effects on the organizations (corresponds to row 4, management orowners primary focus, in Table 2.2). The other similarity is about the limited numberof studies in the literature for measuring the effects of IS on the organizations.152.1.4 A Contingency Theory for IS AssessmentThe studies of Myers focus on the necessity of IS assessment for the effectivemanagement and continuous improvement (Myers et al, 1997). Myers suggests acontingency theory for the IS assessment to guide senior IS managers in selectingappropriate dimensions and measures for their organizations. The purpose of thecontingency theory stems from the goal of providing guidance for an IS assessmentselection strategy.IS managers often lack the tools they need to decide if they are accomplishing theright activities. In addition, there are abundant resources for selecting measures. Inthese circumstances, the theory aims to help IS managers by answering the followingquestions;• What are the appropriate IS success dimensions that should be assessed withineach organizational and external environmental context?,• Once the appropriate dimensions are selected, what are the appropriate measuresto evaluate performance in each dimension, again, given the context of theorganizational and external environment?, and• Finally, how should these IS success dimensions and measures be selected?.The contingency theories propose that different strategies are appropriate fordifferent competitive business settings. They differ from the universal view byemphasizing "it all depends" and they differ from the situation specific view byasserting that there are classes of settings for which strategic generalizations can bemade. Corporate level strategy, organization structure, industry, organization size,business strategy, work group interdependence, culture, incentive system,information intensity of products and/ or services, IS management expertise, IS end-user skills, strategic role of IS, size of IS organization, IS budget size, userparticipation/involvement, history of organization, individual characteristics, task,climate, and location of the responsible executive are presented as potentialcontingency variables.16The theory is summarized in Figure 2.3. IS Success Dimensions and SelectedMeasures, adapted from (Saunders and Jones, 1992), is depicted in the left hand sideof the figure. Saunders and Jones (1992) developed the "IS Function PerformanceEvaluation Model" which was used to describe how measures should be selectedfrom the multiple dimensions of the IS function relative to specific organizationalfactors and based on the perspective of the evaluator. This model provides additionalknowledge to the developing theory for IS assessment. There are considerableoverlaps between the studies of DeLone and McLean (1992) and Saunders and Jones(1992). Several DeLone and McLean IS Success Model?s categories of IS successare represented by one or more of the Saunders and Jones Performance Dimensions.For example, the Saunders and Jones dimensions; "IS impact on strategic direction,""IS contribution to organization's financial performance," "integration of IS andcorporate planning," and "integration with related technologies across otherorganizational units" could be all considered as sub-dimensions of "organizationalimpact." Also, "quality of information outputs" corresponds to "information quality,""user/management attitudes" correspond to "user satisfaction," and "adequacy ofsystem development practices" and "IS operational efficiency" roughly correspond to"system quality." The six dimensions of the DeLone and McLean IS Success Model(1992) are updated with suggested measures from the studies in other disciplinessince 1988. Also, the two additional dimensions (service quality and workgroupimpact) of IS success are strongly mentioned as worthy to be included in the modeland other possible measures are presented. Some of the measures for each IS successdimension are provided and supplemented with the lists collected by DeLone andMcLean IS Success Model (1992) as well. For instance, cost savings, improvedcustomer service, improved productivity, return on investment and increased dataavailability measures are defined for organizational impact dimension.There is ?Selected Contingency Theory Variables? as External EnvironmentalVariables and Organizational Variables in Figure 2.3. External environmentalvariables include industry, competitive environment, culture, economy, availabilityof resources and climate arguments. Organizational variables include mission, size,goals, top management support, maturity of IS function, size of IS function, culture17and IS budget size arguments. It is aimed to select and prioritize of IS successdimensions and select measures for each dimension according to these arguments.IS SUCCESS DIMENSIONS& Selected Measures SELECTEDSERVICE QUALITYCONTINGENCY THEORY. SERVQUALVARIABLESSYSTEM QUALITY. reliability. response time. ease of use. cost benefit analysis. pos