Thursday, September 17, 2009

QEngine, Selenium and PureTest

QEngine, Selenium, PureTest comparison

 

I have tried comparing QEngine , Selenium and PureTest in the limited period of time I had. As you would know all of these tools are automated testing tools.  Being a developer I found all of them easy to install and use.

 

Here is a simple comparison:-

 

QEngine :

The product is now sold by ManageEngine, previously it was owned by AdventNet. You can download a free trial at http://www.manageengine.com/products/qengine/ .

This tool provide some of the unique features like load testing, stress testing, web service testing, performance testing reports, web based test manager etc.

QEngine application acts as a server looking for specific browsers request. This request comes from browser which has QEngine plug-in installed. Thus, it is mandatory to have QEngine plug-in installed on your browser to use this application.

Once you have this setup ready you can do everything you can do with QEngine in the browser itself.

Advantages:

  1. Test case recording/replay
  2. Ajax testing
  3. Load testing
  4. Stress testing
  5. Database testing – monitors db connections
  6. Application server testing – monitors server resources viz. CPU and memory usage
  7. Modem simulation
  8. Regression test scheduling
  9. Web service testing

Limitations:

  1. You cannot use the recorded test cases in any other tool e.g. JUnit
  2. No support for Xpath. It identifies html elements only through tag and its attributes
  3. Not suitable for Web 2.0 applications

Thought I tested this on IE8, I could not install the QEnigine plug-in on FF 3.5.

 

Selenium:

I find easy to use Selenium, may be because I have been working on it for quite a long time now. I used Selenium IDE  and Selenium Core to test applications. You can download these tools from http://seleniumhq.org/download/ .

The only way to record test cases hear is by using Firefox add-on “Selenium IDE”. Once you have recorded test cases you can play them on different browser by using “Selenium Core” or “Selenium RC”. “Selenium Core” core provides a single Web page where you can test the application or view the test cases on the same page. The page is divided into two frames one for Selenium and other for your web application.

Advantages:

  1. Test case recording/replay
  2. Easy customization of recorded test scripts
  3. Ajax testing  - provide easy options like waitForElementPresent command
  4. Java/JUnit integration
  5. Free tool
  6. HTML elements can be location using different ways like Xpath, CSS selectors etc. – very useful in testing if you are using JavaScript frameworks like ExtJS
  7. No need for any external plug-in while testing on different browsers

 

Limitations:

  1. No support for performance testing

 

PureTest

You can download PureTest from http://www.minq.se/products/puretest/ .

Once you install PureTest application, you need to configure your browser proxy to point to PureTest server. After setting  the proxy, any request-response action you do on  the browser get recorded in the script. You can view these scripts in the PureTest application window. It does not support browser interface – you can use browser only for recording.

You can use the recorded script to replay back in the PureTest application, you can edit request or validate response.

Advantages:

  1. You can test your application without any browser.
  2. Can change request parameters
  3. Can validate response
  4. Can record all browser requests including Ajax calls
  5. Free tool
  6. Web crawler included
  7. Very easy to record/playback

 

Limitations:

  1. Cannot test UI elements
  2. No support for performance testing

This can also be used for detecting broken links.

 

 


Sunday, August 30, 2009

Function Point Analysis(FPA)

Function Point Analysis

Following is the brief summery from the IFPUG Counting Practices Manual 4.1.

 

Introduction

IFPUG counting practices manual is usually updated on January month of every year. I am referring here an old version of this manual which was released on January 1999. The practices might have changed since then.

The latest manual is available on IFPUG site only to registered members. Membership is not free.

Definition: Function point analysis (FPA) is a standard method for measuring software development from the user point of view.


Features:

  • FPA quantifies the software system primarily based on logical design
  • Measures functionality which users request/receives
  • Independent of technology used for implementation
  • Simplifies measurement process
  • Consistent process among various projects and organizations


Usage:
 

  • Find estimate cost and resource required for software development
  • To determine size of purchased application package
  • To determine use of application package to the needs of organization
  • Software comparison

Procedure for FPA calculation
 

  1. Determine type of function point count
  2. Identify application boundary
  3. Determine unadjusted function point count
    1. count data function
    2. count transaction function
  4. Determine value adjustment factor
  5. Calculate adjusted function point count

Figure 1: Procedure Diagram

Function Point Count Type

There are three types of function point count:-

  1. Development project function point count - The development project function point count measures the functions provided to the users with the first installation of the software delivered when the project is complete.
     
  2. Enhancement project function point count - The enhancement project function point count measures the modifications to the existing application that add, change, or delete user functions delivered when the project is complete.
     
  3. Application function point count – The application function point count and project count are associated with an installed application. It is also referred to as the baseline or installed function point count. This count provides a measure of the current functions the application provides the user. This number is initialized when the development project function point count is completed. It is updated every time completion of an enhancement project alters the application's functions.

Figure 2: Type of counts

Formula for calculating adjusted function point count changes depending on function point count type (step 1 in procedures).

Counting scope and application boundary

Counting scope means the functionality that will be included in particular function point count.

Application boundary is the border between the system being measured and users. (See fig. below)

Figure 3: Summery diagram for Human Resource Application

 

Unadjusted function point count

Unadjusted function point count (UFPC) is the specific countable functionality provided by the system or project to the users. UFPC is evaluated in terms of what system does and not how it done.

UFPC has two function types:-

  1. Data function
  2. Transactional function

Figure 4: UFPC function type

 

The basic difference is Data functions deals with data and Transactional function deal with processing of data.

a) Count Data Functions:

Data functions represent functionality provided to users by the system to meet internal or external data requirements.

The term file here does not mean file in the traditional data processing sense. In this case, file refers to a logically related group of data and not the physical implementation of those groups of data. Data functions are either Internal Logical Files (ILF) or External Interface Files (EIF).

  1. Internal Logical File (ILF) – An internal logical file is a user identifiable group of logically related data or control information maintained within the boundary of application.

    ILF holds data maintained through one or more elementary processes of the application being counted. Figure 2 shows a group of related employee data maintained within the Human Resources Application.
     
  2. External Interface File (EIF) – An external interface file is a user identifiable group of logically related data or control information reference by the application, but maintained within the boundary of another application.

    The primary intent of EIF is to hold data referenced through one or more elementary process within the boundary or the application counted. Figure 2 shows conversion rate information maintained by the Currency Application and referenced by the Human Resource Application.

An elementary process is the smallest unit of activity that is meaningful to the user(s). E.g. add a new employee to employee list.

Control Information is data that influences an elementary process of the application being counted. It specifies what, when, or how data is to be processed. E.g. schedule payment of employee.

The primary difference between an internal logical file and an external interface file is that an EIF is not maintained by the application being counted, while an ILF is.

b) Count Transactional Functions:

Transactional functions represent the functionality provided to the users by the system to process data. Transactional functions are External Input (EI), External Output (EO) or External Inquiry (EQ).

  1. External Input (EI) - An external input (EI) is an elementary process that processes data or control information that comes from outside the application’s boundary.

    The primary intent of an EI is to maintain one or more ILFs and/or to alter the behavior of the system. The example Figure 2 shows the process of entering employee information into the Human Resources Application.
     
  2. External Output (EO) – An external output (EO) is an elementary process that sends data or control information outside the application’s boundary.

    The primary intent of an external output is to present information to a user through processing logic other than or in addition to the retrieval of data or control information. The processing logic must contain at least one mathematical formula or calculation, or create derived data. An external output may also maintain one or more ILFs and/or alter the behavior of the system. The example on Figure 2 shows the process of producing a report that lists all employees stored in the Human Resources Application.
     
  3. External Inquiry (EQ) - An external inquiry (EQ) is an elementary process that sends data or control information outside the application boundary.

    The primary intent of an external inquiry is to present information to a user through the retrieval of data or control information. The processing logic contains no mathematical formula or calculation, and creates no derived data. No ILF is maintained during the processing, nor is the behavior of the system altered. The example on Figure 2 shows the process of inquiring on employee information (input request) and viewing an employee's information when it appears on a screen (output retrieval).

 

Notes:

  • Count one DET for the capability to send a system response message outside the application boundary to indicate an error occurred during processing, confirm that processing is complete or verify that processing should continue. E.g. validation messages, error messages

Value Adjustment Factor

The value adjustment factor (VAF) indicates the general functionality provided to the user of the application. The VAF is comprised of 14 general system characteristics (GSCs) that assess the general functionality of the application. Each characteristic has associated descriptions that help determine the degree of influence of the characteristic. The degrees of influence range on a scale of zero to five, from no influence to strong influence.

Adjusted Function Point Count

The adjusted function point count is calculated using a specific formula for a development project, enhancement project, or application (system baseline) function point count.

Software cost estimation using the Function Point (FP) Method

 

To accomplish a Function Point estimation, we need to analyze the information domain of the software. That is, what is required of the software and what the output would be, to give a weighting factor. After this analysis, we use a given formula to calculate the amount of FP that exist in the software, then apply the cost per FP to arrive at a final estimation. All of this is done as follows.

 

Weighting Factor Estimation:

 

Measurement parameter         count        simple      average      complex   

 

Number of user inputs                             x       3                4                   6        =

 

Number of user outputs                           x       4               5                   7        =   

 

Number of user inquires                          x       3               4                   6        =   

 

Number of files                                        x       7              10                 15      = 

Number of external interfaces                 x       5                7                  10      = 

 

Total 

 

 

After getting a Total, we now use the following formula to get the FP:

 

FP = Total x (0.65 + 0.01 x  3 Fi )

 

Where the constants are empirical values and Fi is obtained from the following 14 points evaluation. 

 

Rate each factor on a scale of 0 to 5  

 

 

No         Incidental           Moderate                 Average      Significant       Essential

 

  1. Does the system require reliable backup and recovery?

 

  1. Are data communications required?

 

  1. Are there distributed processing functions?

 

  1. Is performance critical?

 

  1. Will the system run in an existing, heavily utilized environment?

 

  1. Does the system require on-line data entry?

 

  1. Does the on-line data entry require the input transaction to be built over multiple screens or operations?

 

  1. Are the master files updated on-line?

 

  1. Are the inputs, outputs, files, or inquiries complex?

 

  1. Is the internal processing complex?

 

  1. Is the code designed to be reusable?

 

  1. Are conversion and installation included in the design?

 

  1. Is the system designed for multiple installations in different organizations?

 

  1. Is the application designed to facilitate change and ease of use by the user?

 

When we evaluate the above 14 points and sum the evaluation,  we would add a complexity adjustment factor of 1.17.

 

For Example:

 

Problem statement:

 

A software package is to be developed for a CAD.

 

The CAD software will accept 2 and 3 dimensional geometric data from an engineer. The engineer will interact and control the CAD system through a user interface that will exhibit characteristics of good human-machine interface design. All geometric data and other supporting information will be maintained in a CAD database. Design analysis modules will be developed to produce required output which will be displayed on a variety of graphics devices. The software will be designed to control and interact with peripheral devices that include a mouse, digitizer and laser printer.

 

This statement needs to be expanded when requirement analysis (elicitation) is done to get a complete picture of the system. After this was done the following is obtained .

 

If we  had to develop a software  (name ALPHA) and after analyzing the requirements, etc. we end up with the following Weighted Factor Estimation. Assuming we are using average, we get the following:

 

 

             

 

Weighting Factor Estimation:

 

Measurement parameter         count        simple      average      complex   

 



Number of user inputs                24            x                        4                           =   96

 



Number of users                           16           x                       5                           =   80

 



Number of user inquires               22           x                       4                           =    88

 



Number of files                             4           x                     10                          =  40

 



Number of external interfaces       2          x                       7                           =  14

 


Total  --------------------------------------------------------------------------------------->      318

 

And for the 14 points we had plus the complexity adjustment factor:

 

Factor                                                                  Value

 

  1. Backup & recovery                                           4
  2. Data communications                                       2
  3. Distributed processing                                      0
  4. Performance critical                                          4
  5. Existing operating environment                        3
  6. On-line data entry                                              4
  7. Input transaction over multiple screens             5                                
  8. Master files updated on-line                              3
  9. Information domain values complex                 5
  10. Internal processing complex                              5
  11. Code designed for reuse                                     4
  12. Conversion/installation in design                       3
  13. Multiple installations                                          5
  14. Application designed for change                        5
  15. Complexity adjustment factor                             1.17  

 

3 F15                                                                               53.17

 

Recall the formula:  FP = Total x (0.65 + 0.01 x  3 Fi )

 

We now have   FP = 318 x (0.65 + 0.01 x 53.17)

                               =  375

 

Now if we have the following assumptions:

 

  1. The organization produces on an average 6.5 FP per month

 

  1. And labor costs is $8000.00 per month

 

We can now make an estimate of the software:

 

At 8000 per month with 6.5 FP per month, then the cost per FP is $(8000/6.5) which is approximately $1230.

 

For 375 FP the project will cost $(1230 x 375) . $461,000.

 

NOTE: The information above is collected from various sites. Some information might be copyrighted.

 


Was the information useful?

Followers