Test Details

Owned by Sanjay Sahani
Last updated: Feb 04, 2015
2 min read

Test Objectives

  1. To identify the size of data for which RMsis can be used without compromising on the system usability / responsiveness. 
  2. Establish the system performance from a concurrency viewpoint, i.e. how the performance degrades with concurrent requests.
  3. Establish that the system is able to perform without exhausting the memory

Assumptions

We have made the following assumptions, while simulating concurrency. These assumptions are based on our insights into few customers and perceive that these represent a typical scenario inside a large organization.

  • For every 1000 consumers of Requirements, we are assuming that there will be 100 authors of Requirements. Assuming proportional read / write requests, we will have the ratio of 10:1.
  • Assuming that every user makes one request per 50 seconds under peak load conditions.
  • So when all users are logged in and active, we expect to get about 20 requests / second.

 

Note

  • This data has been designed in such a way that most of the users are concentrated in one large project. This was done to establish the limits to which RMsis 1.8.0 can be pushed for large projects and large number of users in a singular project.
  • However, the system will perform much better, when users are distributed across multiple smaller sized projects.

 

Test Data for Concurrency Tests

Concurrency tests were run on an installation with reasonably large data set. The details are as follows:

  • 1 project with 25,000 Planned Requirements, 25,000 unplanned Requirements, 20 projects with 3000 requirements each.
  • 1 project with 50,000 Issues, 20 projects with 5000 issues each.
  • 1 project with 25,000 Test Cases, 20 projects with 5000 Test Cases each.
  • Concurrent users are distributed across projects in the proportion of requirements.

Test Data for Functional Performance vs. Project Size

In order to cover the entire spectrum for functional response time, test data was created for 2 typical installations:

  • Very small installation, typical of a small team with less than 10 users. The details of this installation are
    • 1 project with 1000 Requirements, 9 projects with 100 requirements each.
    • 1 project with 1000 Issues, 9 projects with 100 issues each.
    • 1 project with 1000 Test Case, 9 projects with 100 Test Cases each.
  • Very large installation, where we have tried to create an environment which we would like to target for a reasonable response..
    • 1 project with 25000 requirements, 49 projects with 20000 Requirements each.
    • 60 projects (10 projects inactive in RMsis) with 20000 Issues each.
    • 50 projects with 20000 Test Cases each.

Intermediate data points may be created at a later date.

Test Conditions

  1. i7, desktop class machine with RMsis, JIRA and Database running on the same machine.
  2. MySQL Database.