HomeUser GuidePrinter Friendly Version

User Guide

1. Overview

1.1. Overview

This documentation is targeted towards the persons interested in learning how to use Test Collab

Test Collab is a modern test management tool that offers a complete platform for your application's testing. It integrates with all popular bug trackers and test automation tools. It also supports agile methodology, requirements management, test plans and scheduling. 

This product is not limited to IT industry, due to its flexibility it can be easily adapted by any industry that has QA needs. 

Throughout your reading, we will try to familiarize you with the application concepts. 

With Test Collab one can create projects, manage project's requirements, set milestones, define test suites, test cases, create test plans, assign test executions, get them running, and analyze the results. 

Test Collab also supports two way integration with issue tracking systems like JIRA and Redmine. It can also be integrated with any test automation tool. 

As we go on we will discuss all these features in detail. 

Starting with definitions of key terms, one by one. 

Project

Projects are main organizational units of Test Collab. A project can be any software, website or a product. Every project has its own members, test suites, test cases, requirements, test plans and test executions.

Apart from administrative tasks, anything you will do in this application will be under a project. Simply stated project encapsulates all the objects and functionalities that become the basis for test case management system.  

Requirements 

Project requirements , project specifications , project objectives or simply requirements form the basis of a product. Analysts, developers, testers all depend on these requirements  as they state what features the product should have and how they should be implemented. 

Milestone 

Milestones mark key points in a project so you can have a bird-eye view over things that need to be done, when and who is responsible. Milestone can be a single day event or a time period with start and due dates. 

Set the milestone, as it will help team members in knowing what the goals are, who is responsible and what the deadlines are.  

Test Suite

A test suite is a collection of test cases that are intended to be used to test a process or software program to show that it has some specified set of behaviors.

A test suite often contains detailed instructions or goals for each collection of test cases and information on the system configuration to be used during testing.

A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests. More details.

Test Case 

test case in software engineering is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not.

It may take many test cases to determine that a software program or system is considered sufficiently scrutinized to be released.

Test cases are often referred to as test scripts, particularly when written. Written test cases are usually collected into test suites. Source.

A test case is

  • A single, 
  • A group, or 
  • A grouping of increments 

a software tester writes to demonstrate how an application functions.

These increments or "steps" the tester writes to map out the characteristics of how a software program will behave when it is executed.

The core entity around which the all processes and sub processes are designed is a test case. 

Test Plans 

A test plan documents the strategy that will be used to verify and ensure that a product or system meets its design specifications and other requirements.

Test Execution 

Test Execution is an entity which groups together the test cases and their testers. They can be easily assigned and monitored.

Here testers are team members of a project, or remote executers defined under the application.

 

Read on for further details.

2. Dashboard

2.1. Dashboard

The dashboard provides a hawk eye on overall performance enabling the user to monitor his Projects by showing the following:

  • Graphical representation of the results of test executed during a particular period
  • The latest activities performed for the projects
  • Upcoming milestones

Graph

The graph covers all executed test cases pertaining to user's projects, it depicts the following:

  • Number of test cases
  • Time taken or consumed in their execution
  • Indication of passed and failed cases

List of Activities

 

The chronologically descending ordered list will cover following:

  • Individual activity performed (pertaining to Projects / Suites / Cases / Requirements / Milestones / Test Plans / Executions etc)
  • Name of user who performed the activity
  • Time spent since the activity was performed

Upcoming Milestones

When there are one or more pre-existing milestones that are yet to be achieved, then for each milestone following is displayed: 

  • Total number of "Test Executions" under the milestone
  • Progress of milestone shown as percentage (graphically) and number of pending Test Executions
  • Start Date and Due Date
  • Person responsible for the milestone

Accessing Projects 

User can choose an operation for his projects by clicking on the Projects tab.

Quick Links

  

If the user is an administrator, then the additional features available through dashboard would include the following:

  • All the projects under the application will be covered on the graph and on the activities list
  • An add new project option, is available under Quick Links  and via Projects tab
  • Access to Users Management, Roles Management and Settings sections

If the user is not an administrator then the details of projects would be limited to the ones he is member of.

Throughout the application, on top right corner of the screen, the user will see links to:

  • Access his Profile
  • Get Help
  • Software Info
  • Logout of the application
  • Jump between dashboard and different projects

.

3. Projects Management

3.1. Project Management

A Project can be understood as an entity that encapsulates all the objects and functionalities that become the basis for test case management system. 

These projects include Test Suites and Test Cases at a very basic level. 

In TestCollab, projects also include Milestones , Test Plans and Test Executions

The test cases can be linked to the project's requirements the requirements can be maintained within Test Collab, they can be fetched from an issue manager or they can be linked from an external source. 

Archiving of projects is also possible. 

Adding, editing, archiving and deleting projects can only be done by administrators.  

Add a New Project 

Only users with administrator privileges can add a new project to the application. 

 

Projects' List

For an administrator, the list of projects provides the details on all projects under the application and which are not archived.

For a non administrator, the list of projects would be limited to projects for which he is a member of.

For each project in the list the details include the following:

  • Total time spent during the execution of various test cases under the project
  • Average time/execution, and average success % per execution are calculated on the basis of past "Test Executions'" history
  • Number of test cases, test executions, defects reported under the project
  • Last activity performed for the project

The links to perform various operations like View / Edit / Archive / Delete are shown when the cursor is hovered over the project details.

Project's Overview

Helping user keep track of the components of a project discussed above, project overview page gives analytical overview of the entire project by mainly the :

  • Graphical representation of the results of test executions under the project during a particular period
  • Latest activities performed

Graph


The graph covers all the executed test cases pertaining to project, it depicts the following:

  • Number of test cases
  • Time taken or consumed in their execution
  • Indication of passed and failed cases 

Test Metrics


Metrics provides collective and tabular representation of the key statistics for analysis.

List of Activities

The chronologically descending ordered list will cover following:

  • Individual activity performed (i.e. Projects / Suites / Cases / Requirements / Milestones / Test Plans / Executions etc)
  • Name of user who performed the activity
  • Time spent since the activity was performed
Quick Links

Add project members option is available only for administrators.

The tabs on the page provide access to the following:

  • Test Suites and Test Cases
  • Milestones
  • Test Executions and Plans
  • Requirements (if enabled for the project)
  • Reports
  • Project Specific Settings (for administrators only)

Archived Projects

Archived Projects' list can be viewed by selecting the Archived option under the Projects Tab.

3.2. Project Members

Members are users (not application administrators) with a role assigned to them.

Administrators always are members of each and every project in TestCollab.

These members once added then depending on the roles assigned, can 

  • Create, edit import, export or manage the project requirements (if in-built requirements management is enabled)
  • Create, edit, import, export or manage test suites and cases
  • Create, edit, delete and be held responsible for a milestone
  • Create, edit, delete or be assigned a test plan or a test execution
  • Even work in collaborative way and comment and do other operations (as authorized for) on the tests executed or set it to be executed under a test execution.
  • Generate and view reports

If an issue manager is configured to be associated with application, then every member's profile can be linked to that issue manager; steps for this are covered under user management.

When a user's profile is linked, any reporting via the issue manager will be done automatically, if opted by tester, this is in cases of failure of a Test Execution.

Any operation on team member's data can only be performed by an administrator.

Add a Team Member

Following is the form used to enter new member's details:

Administrator has to:

  • Select the user(s) from a list, and
  • Role that can be assigned to them

If email notification for this action has been enabled, then an email would be sent to the new members.

Project Members' List

On the list, for each project member:

  • Name of the user
  • His role for the selected project
  • Links to edit or delete the member, will be shown

A link to add a new team member will also exist.

Administrator also has an option to select the number of records to be shown per page.

If administrator opts to edit a members's info, then a form similar to the 'add member' is shown.

3.3. Custom Fields

Custom fields as the name suggests are the fields that make it possible to define additional properties for an entity.

In Test Collab the custom fields can be defined for:

  • Test Cases
  • Test Executions

Custom fields can also be seen as additional parameters that refine the arguments between the person defining the Test Execution and the person who actually executes it. They may be set to control the execution of a test.

If custom fields are present under the project, then they can be used in following ways:

  • To define additional properties to a test case and test execution under that project
  • To map their values with the values of issue manager fields that are selected while configuring issue manager for project so that the reported issues hold the mapped custom field's values
  • To define the skip conditions for an individual test case or a suite altogether. While defining test suites and test cases, a user can select a custom field of type "Option box" and "Multiple Choice Option box" to be used for a condition and select the value for it to drive that condition.
  • Control the execution, as they may be the part of skip conditions
  • As source of communicating a fixed set of information between a tester and others
  • As part of the analytical data

Any number of custom fields can exist under a project.

Following are the Properties of a custom field:

  • Custom field defined for
  • Name
  • Label (shown to users using it)
  • Type (to govern the user interface)
  • Required (whether it will be mandatory or not)
  • Default value

Adding, Editing, Deleting of custom fields can only be done by an administrator. 

Add a Custom Field

Following is the form used to enter a new field's details.

The user will be required to enter/select the following:

  • The entity for which custom field is being defined
  • The name for the new custom field
  • Its label
  • Its type, where type can be
    • Text
    • Multiline Text
    • Option Box
    • Multiple Choice Option Box
    • Date
  • Whether it will be required to enter data for the field, or not
  • Its
    • Default value (for Text), or
    • List of options (for Option Box/Multiple Choice Option Box) available

Show on search option can be selected, if this field will be required to be used as a filter for search criteria.

Custom field for, name, label, and type are mandatory.

Edit a Custom Field

A form similar to add Custom Field is available to edit a custom field and the user can update any of the information.

When custom fields name, label, and type are available for the field, then record can be updated.

Custom Fields' List

 

In addition to the list of pre-existing custom fields, the links to Edit/Delete them, and a button to add new field will also be available.

The administrator also has an option to select the number of records to be shown per page will be available through a drop down.

3.4. Field Mapping

This feature provides you an option of getting the default and custom fields used in Test Collab project to be mapped with the associated issue manager fields through the values. By mapping these field values you make sure that the values for the issues reported in the issue manager are the ones you wanted to be when you or your team members were selecting their counter parts for test cases or test executions in Test Collab.

Mapping is basically done between the fields you have selected while configuring the issue manager for the project, and the default and custom fields that exist in the project for test cases and test executions, i.e. you will have to decide that a particular field selected for issue manager is to be mapped with which Test Collab field. When you select the Test Collab project field, you will then be presented with option to provide value (as it should be in reported issue) to map for each of the options you have defined with your field.

Supported field types for the purpose are option box, multiple choice option box and radio buttons (fields that provide a set of possible values).

Here is an example of field mapping: 

For instance if while adding or editing an execution I opted the platform to be "Linux" then at the time the relevant issue is reported the value for the mapped field would be set as "debian"

3.5. Milestones Management

Add a New Milestone

When an authorized user opts to add a new milestone, then a form will be shown to enter the information.

All the fields except responsible user are mandatory.

Milestones, once created, can be associated with Test Executions.

Edit a Milestone

When an authorized user opts to edit a pre-existing milestone, then a form similar to add milestone will be shown and user can update any part of the information.

Milestones' List

The milestones' list screen lets an authorized user manage the milestones.

The user can see the list of existing milestones under the selected project, along with their relevant details.

For each milestone, following is shown:

  • Progress of the milestone in percentage (shown graphically)
  • Total number of "Test Executions", and pending "Test Executions"
  • Start date, and due date of the milestone
  • Name of the person responsible for milestone

The links to perform various operations like View/Edit/Archive/Delete are shown when the cursor hovers over the milestone's details

If email notifications for milestone's due date approaching, its completion, or its delay have been enabled by an administrator, then the email would be sent. 

Viewing Milestone Details


Milestone details cover graphically represented statistical information related to milestone like

  • Percentage of completion and percentage of passed tests
  • General information related to a milestone

A link to allocate new Test Executions to the milestone is also available for authorized users.

Details of allocated Test Execution under the milestone are also shown.

Hovering the cursor over a Test Execution would provide links for user to View/Edit/Assign/Delete the Test Execution.

Archived Milestones

Archived Milestones' list can be viewed by selecting Archived option under Milestones Tab.

4. User Management

4.1. Users Management

Different ranks exist in the application that decide what kind of privileges a user will have.

Ranks

Types of ranks available under the application are

  • User,
  • Administrator, and
  • Super Administrator

Here a User has the access defined or limited by either an administrator or a super administrator.

  • A user can normally be added to the team of members working on a Project in application. 
  • He can perform the operations that are defined as allowed for the Role he is assigned for the project, and 
  • A user can access and change the information related to his profile. 

Administrator is a rank at the middle level.

An administrator can

  • Manage users,
  • Manage roles,
  • Manage projects,
  • Change the application wide settings

Super Administrator, can in fact

  • Change the ranks assigned to other users of the application
    • He can either Upgrade a user to act as an administrator, or
    • He can degrade an administrator to become a user with limited rights
  • Update the plan (if application is hosted on testcollab cloud)
  • Cancel the account (if application is hosted on testcollab cloud)
User management tasks like adding/editing/deleting can only be performed by an administrator.

Add a New User

 

A form is available to enter new user's information like

  • E-mail
  • Password along with its confirmation
  • First name
  • Last name (all the above are mandatory fields), and
  • Also an option to upload a profile picture (avatar) for new user, will be there
 
  • The password should be of prescribed length, and
  • The password should be confirmed correctly

If an administrator has enabled the email notification for this action, then the same would be sent.

Edit a User's Record

The form used to update the selected user's record.

 

If selected user is an administrator then on Edit User or Edit Profile screen an additional option to subscribe to all the mails generated for various activities in the system will be available for the user whose profile is being edited. Once opted, that user will receive a notification of any activity (treated as notifiable by system) via email.

If issue manager has been configured for the application, then a link to associate the selected user to the issue manager account will be shown.

Linking User's Profile with an Issue Manager

This link will enable the new issues to be created on user's behalf. In absence of this setting, the issues will be created using the default account set as reporter by the administrator.

If administrator opts to link the user account issue manager's account with user, a separate form will be provided to let the administrator enter API key for the issue manager.

 

Users' List

 

For super administrator, an additional option available would be to change the rank of user.

View a User's Details

On selecting view option for a user, the screen shown is

 

On the screen general information related to user will be shown which includes

  • Name
  • Email
  • The date on which profile was created
  • The date the user had last signed-in to the application, and
  • If the user's profile has been linked to issue manager, then the unique API key associated with user's account will also be shown

A link to edit the user's profile will be available.

If selected user is not an administrator then the under mentioned details would be limited to the projects he is member of.

  • List of the projects showing the name of project, followed by the role a user has been assigned for the project.
  • List of latest activities pertaining to the projects

4.2. Roles Management

Roles can be understood as the privileges assigned to a user for a Project.

These roles define whether a user assigned the said role, will be authorized to perform certain actions or not.

Roles are categorized logically in sections that relate to different modules/sub modules within the application.

Roles are managed only by an administrator.

To know more about the difference between super administrator, administrator and a user, visit our User Management section.

Add a New Role

Only users with administrator privileges can add a new role to the application.

 

A form is available to enter new role's name.

Also available will be a set of permissions that a user having that role can be granted.

The sets of assignable permissions would be categorized in logical sections that relate to different modules of the application, like

All major sections have a permission to Manage that module or section. A check on "Manage" would mean that permissions have been granted for all actions under that module/section/entity.

The administrator also has an option, to restrict a tester from pausing a running test, and if a user does not have this permission, then he will also not be able to switch to another test execution without finishing an already started test execution.

Edit a Role

The form to edit a role is similar to that used to add a role.

Roles' List

The list of roles is accessible only to the user with a privilege of an administrator

 

4.3. Profile Management

A logged-in user has got options to view and manage his profiles. 

 

View Profile

On selecting My Profile option the screen shown is 

 

 

A link to edit his profile will be available.

If selected user is not an administrator then the under mentioned details would be limited to the projects he is member of

  • List of the projects showing the name of project, followed by the role a user has been assigned for the project.
  • List of user's latest activities pertaining to the projects 

Edit Profile

If a logged-in user has opted to edit his profile, then he will be redirected to the form used to update his profile information. 

 

User can change his

  • E-mail,
  • First name,
  • Last name,
  • Subscribe to all emails generated for various activities (if and only if logged in user is an administrator)
  • Avatar (profile pic), and if not already uploaded, then an option to upload new avatar, and 

If Subscribe to all emails generated for various activities is selected then the user will also receive notifications on all major activities performed within the application through email.

 

If issue manager has been configured for the application, then a link to associate his profile to the issue manager account will be shown.

Linking Profile with an Issue Manager

This link will enable the new issues to be created on user's behalf. In absence of this setting the issues will be created using the default account set as reporter by the administrator

 

If user opts to link his issue manager's account with his profile, a separate form will be provided to let the user enter his credentials for the issue manager. The credentials can be an API key for that user revealed by issue manager or the username and password used by user to access the issue manager. The entry fields will depend on the issue manager being used, because bug trackers like JIRA and Fogbugz do not provide API key.

 

 

 

Please note - TestCollab does not validate the access credentials entered by user at this stage, unlike as done at the time of issue manager integration. If the credentials are incorrect then the bugs will not be reported in the name of the selected user.

The process of linking a user account with the integrated issue manager can also be performed by application's administrator(s).

5. Test Suites and Test Cases Management

5.1. Test Suite and Test Case Manager

Test Suites and Test Cases Manager

The manager screen provides an interface which enables user to perform all kinds of diverse operations for test suites and test cases.

 

Operations like viewing, prioritizing, sorting, copying and re-grouping tests can be performed using this screen.

Additional features include: 

  • Importing and exporting test suites in XML format, and 
  • Importing and exporting test cases in CSV format, all the operations are possible from a single screen 
  • Importing test cases from Excel files and from other projects 

User can filter test cases on the basis of suite's title, test case's title, test case's priority, tags applied, requirements linked with them while defining cases, and custom fields that are defined to be used for test cases.

When test suites and test cases are present under a project, they will be shown in a tree like structure on left pane. 

The test suites and test cases under a suite can be expanded or collapsed by either clicking onto the test suite or by clicking on the icon provided for the purpose.

Other operations that can be performed are :  

  • Add new test suites and new test cases
  • Copy test suites (within same project) and cases (primarily within same test suite)
  • Import (test suites) from other projects or from XML, CSV or Excel
  • Export (test suites) as XML or CSV
  • Filter (test cases) by Tags, Requirements, Priority and custom fields defined within the project for test cases
  • Link all test cases to requirement(s)

Context menus available for suites and cases 

While all options available through this menu are straight forward, the option of linking all test cases to requirements mean that when you click on it you will be shown a window where you can enter a comma separated Ids of requirements that you want to be linked with all test cases under selected suite.

 

When a test case is selected the details of the test case will be shown into the right pane, covering

  • Test case title
  • Test case Id
  • Test suite name
  • Description
  • Steps
  • Expected results
  • Attachments
  • Tags 
  • Linked requirements (if enabled for project), and
  • Execution History

 The history of executions (if any) for selected test will be shown in tabular format.

It would be possible to navigate through the peer test cases under the selected suite, edit or delete current test case using the links on right pane. If a recent change has been made in the suite tree (hierarchy) as a result of drag and drop of test suites and cases, then a page refresh is required to have proper navigation.

User has options to:

  • Rearrange (prioritize and sort) the test cases under a suite
  • User can even move a test case from one suite to another by simply dragging
  • More than one case can be selected (for moving) by holding Ctrl key at the time of selecting them

 

When User has opted to filter the test suites and cases then he can simply key in the search criteria in the respective boxes he wants to filter the list on the basis of, once he starts keying the related options will be suggested and appropriate option can be selected.

 

 

Once the filter is applied, the list of test cases and their respective suites will be restricted to the criteria selected.

When filter is applied, Reset option can be used to remove the applied filters and show all cases and suites.

5.2. Test Suites Management

Add a Test Suite

When an authorized user opts to add a new test suite, then a form is shown to enter new test suite's information.

 

User will be required to provide title of the test suite.

Options to add description and to assign a parent suite for suite being newly created are available, and for assigning a parent suite user can select one from list of pre-existing suites.

If custom fields of type "Option box" and "Multiple Choice Option box" are present under the project, then they can be used to define the skip conditions, for this user can select a custom field to be used in condition and select the value for it to drive the condition.

Skip Conditions are used to auto-skip test case(s) from execution which are not required or cannot run on a specific configuration. For example, if for a test suite which need not to run in local environment, then a custom field named Environment can be created with different options like local, staging, production etc., then user can add skip condition for local environment for that particular test suite. Similarly user can apply multiple skip conditions for each test suite.

Here, title is the only mandatory field.

Edit a Test Suite

The form, used to update the selected test suite's record gives the option to change the test suites title, description and its parent.

Import Test Suites from Other Projects

This feature helps the user import test suites from other pre-existing projects.

The option is available under Import menu on Test Suites & Cases tab.

If an authorized user opts to import the test suites from other project(s), then a form like this will be shown.

 

If there are other projects that have test suites created/copied under them, then their list will be on under project's drop down. When a project is selected then typing in the suites' selector box shows up the names of suites that match the criteria entered. 

As an option one can click on the "Select from tree" icon, this will show up the suites' list to select suites from the tree shown.

 

User can select one or more suites he wishes to import, and will have option

  • To copy the selected test suite into current project, or
  • To link the selected test suite with current project
This helps in grouping reusable cases and saves suites' creation and management time

Difference between linking and copying

Linking a test suite will only 'link' the original test suite user has created without creating any copy of it. This feature is useful when user wants to share some common tests between his projects.

When a suite is linked then it can visually identified by a separate icon  and there are some restrictions on the operations that can be performed over a linked suite in target project:

  • It cannot be edited here
  • Its child suites and cases cannot be edited or deleted
  • Its child suites and cases cannot be moved out of it
  • No other suite or case can be moved in this linked suite
However, the linked suite can be moved and made a child of other existing not-linked suite.
 
There is also an option to Unlink the linked suite.
 
Copy function, is used to duplicate all child suites and test cases under selected test suites from the source project to the destination project. This is recommended when there is some level of similarity in projects but user intents to alter a test suite differently for destination project.

Copying means any change in suite or cases made in this (target) project will have no effect onto the project (source) from which it was copied and vice verse.

Exporting Test Suites

This lets a user export a test suite to a CSV file.

This helps in grouping reusable cases reducing redundancy, and saves suites' creation and management time, when a suite with similar cases is required elsewhere.

The option for exporting to a CSV is available under the Export menu on Test Suites & Cases tab.

If a user opts to export the test suite, then a form like this will be shown.

On this page user can export the suite by selecting its name, once the name is selected, a CSV having all the cases under the suite will be generated and will be available for download on to the user's machine.

Please note that only the cases under selected suite will be exported, and its child suites will not be exported. For complete data, use XML export.

5.3. Exporting and Importing Suites in XML format

To help improve the portability and re-usability of test suites and test cases, Test Collab supports exporting and importing the test suites in XML format. 

With the XML export feature you can get the entire test suite tree i.e. the test suite along with all its children suites and test cases exported from an existing project / instance of Test Collab.

Similarly porting the test suite tree from other sources becomes very easy, the sources may be manually maintained test cases or other test case management systems.

Mentioned below are the steps to export the suite's tree : 

Export as XML

Here is the suite (to be exported) shown on Test Suites & Cases manager : 

Click on Export

 On Export Options page, select "Export as XML"   

On XML export page, select the suite to be exported, by either typing its name in the given selector, or by selecting it from the tree that's shown when you click "Select from tree", finally click "Export" button, an XML file will be generated having the entire hierarchy for the selected suite, 

You can get an XML file that has the structure similar to the file that has been attached with this document and XML Structure section. 

The exported XML is ready to be imported in any system that supports the format. 

Import XML

If you have a suite tree ready to be imported in Test Collab that conforms to the supported structure (please refer the attached file and XML Structure section for this) , then you can easily get it imported into project.

Before you proceed ahead there are a few things that need to be taken care of: 

  • Make sure that the file you are using for import has the same syntax (Markdown / HTML / Plain text) as the one being used for your Test Collab's instance.
  • Check the size of file that is being uploaded, as the upload limit depends on the PHP settings done on your hosting server. 
  • All the suites in XML must have their Title and test cases must have both Title and Priority as these are the mandatory fields

 

On Test Suites & Cases tab select Import , On Import Options page, select "Import from XML"

On "Import from XML" page you have options of getting the entire tree introduced under a pre-existing suite of current project or you can also opt to get the suite's tree imported at the root level. 

To select a pre-existing suite either type its name in the given selector, or select it from the tree that's shown when you click "Select from tree".

You may leave the suite name empty if you want to import the test cases at the root level. 

Select the XML file to be uploaded by clicking the relevant box next to "Select XML file".

Update Existing - If you are importing some or all suites / cases with titles that have already been introduced in the current project and are at the same level in suites' hierarchy at which you are planning to import them this time, then you can opt to "Update Duplicates", this means that if there is any change in a test case's structure in XML, then that change will be reflected after import. Please note, here the updates will be directly dependent on the test suite's and test case's title.

If "Update Duplicates" is not selected and during import it is found that there is a duplication in suite's name, then that suite will simply not be imported at all, because in Test Collab there can only be one suite with a name when it comes to a particular level under a particular test suite (or at root level whatever the case is).

After selecting the XML, and setting the other options available on the screen, when you click the "Import" button, you will be redirected to suites manager page, where you can see the imported suite's tree.

XML Structure when single text field is used for test case steps.

<Suites>

<Suite>

<title></title>

<description></description>

<TestCases>

<TestCase>

<title></title>

<description></description>

<description_html></description_html>

<steps></steps>

<expected_result></expected_result>

<expected_result_html></expected_result_html>

<priority></priority>

<sort_order></sort_order>

<Attachments>

<Attachment>

<filename></filename>

<mime></mime>

<downloadUrl></downloadUrl>

</Attachment>

<Attachment> Other attachment's info following the same structure </Attachment>

</Attachments>

</TestCase>

<TestCase> Other test case's info following the same structure </TestCase>

</TestCases>

</Suite>

<Suite> Other test suite's info following the same structure </Suite>

</Suites>

XML Structure when separate text fields are used for test case steps.

 

<Suites>

<Suite>

<title></title>

<description></description>

<TestCases>

<TestCase>

<title></title>

<description></description>

<description_html></description_html>

<steps></steps>

<multi_steps>

<step>

<step></step>

<step_html></step_html>

<expected_result></expected_result>

<expected_result_html></expected_result_html>

</step>

<step>

<step></step>

<step_html></step_html>

<expected_result></expected_result>

<expected_result_html></expected_result_html>

</step>

<step>

<step></step>

<step_html></step_html>

<expected_result></expected_result>

<expected_result_html></expected_result_html>

</step>

</multi_steps>

<expected_result></expected_result>

<expected_result_html></expected_result_html>

<priority></priority>

<steps_html></steps_html>

<tags></tags>

<sort_order></sort_order>

<Attachments>

<Attachment>

<filename></filename>

<mime></mime>

<downloadUrl></downloadUrl>

</Attachment>

</Attachments>

</TestCase>

</TestCases>

</Suite>

</Suites>

5.4. Importing Test Cases Using Excel Spreadsheets

Test Collab users can import the test cases data into a project under a test suite using a Excel Spreadsheet. 

This helps in creating reusable cases, and saves creation and management time for cases that have already been defined. 

The spreadsheet to be used for importing the cases must follow a structure and some basic rules that are also related to test case creation. 

First of all make sure that the file you are using for import has the same syntax (Markdown / HTML / Plain text) as the one being used for your Test Collab's instance.

Secondly check the size of file that is being uploaded, as the upload limit depends on the PHP settings done on your hosting server. 

The spreadsheet can contain the data for following columns: 

  • Title 
  • Description 
  • Steps 
  • Expected Result 
  • Priority 

As they are all part of a test case defined in the application. 

Off these, only Title and Priority are mandatory. 

For Priority, the allowed values are 

  • Low 
  • Normal, and 
  • High  

To separate 2 steps under steps column leave a gap of a line between them. 

In Test Collab, while creating a project you get an option to keep test case steps in single text field; or to have separate text fields for them so that expected results can also be maintained individually, two sample Excel files have been attached with document and these can be referred to create test cases that can then be imported into the application. 

When Import menu under Test Suites & Cases tab is used then an authorized user gets the options as shown below: 

Opting to import the test cases using a Excel spreadsheet, a form like this will be shown: 

 

On the form user will see: 

  • A test suite selector, 
  • An uploader to upload the Excel file (having cases to be imported for selected test suite), and 
  • A box that can be checked if the first row of spreadsheet represents headers, or defined fields in the table of cases being imported 

A test suite selector lists the test suites that exist under the selected project, typing in the suites' selector box shows up the names of suites that match the criteria entered. 

As an option one can also click on the "Select from tree" icon, this will show up the suites' list to select suites from the tree. 

User will be required to select the suite for which he wants the cases to be imported. Empty suites will also appear in the list, to enable the user to import data for them.

Once suite is selected and file to upload has been chosen, user can proceed by clicking import button.

Next screen will allow the user to select the field used in Test Collab individually that maps to a spreadsheet column (name shown as label for drop down). 

When all mandatory fields (for a test case) have been mapped, then a process to merge the data from spreadsheet into the test suite will start. If there are any issues while import then the same will be reported on screen. 

Please note that all the test cases present in spreadsheet used will be imported under a single (selected) test suite. 

5.5. Test Cases Management

Add a Test Case

In this article we are going to cover the main entity around which the entire test case management tool is designed and working.

The main constituents of a test case are its steps which should be clearly defined. Once its steps have been defined, expected outcome also needs to be ascertained.

In order for test execution to be analyzed, the steps and result should be clearly defined.

Test cases can be independent, which means having least or no coupling with other test cases, or they can be cascading where the outcome of one test case becomes pre-condition for other test case.

When an authorized user has opted to add a test case, then a form to enter new test case information will be shown.

 

User can provide information like 

  • Association of the new case with a pre-existing test suite
  • Title of the case which should be brief but self explanatory
  • Priority of the test case is to be selected from a set of options like - Low,Normal and High
  • Description, though not mandatory but is one of the prime sources of information to tester, the description can be details about test case or any pre condition which may be some kind of settings or scenario that should exist prior to a test has initiated
  • Steps, these are the most important set of one or more instructions to execute the tests, arranged in a logical sequence
    In the editor provided for steps, he will have option to add the defined steps that are included under the test case, 
    • For each step, user should be clear in defining the part of the procedure that step is expected to be used for, 
    • To separate 2 steps user should provide a gap of 1 line by hitting Return (Enter) key twice, 
    • To provide the properties (format) for text being used for each step user will have set of icons associated with the text editor. 
    • When Markdown editor type is being used then the steps that have been defined as reusable steps can be easily imported for the test case by using the given icon. (user can also define new set of reusable steps, if they don't exist by using the same icon)
    • Once the user opts to import reusable steps, a window will popup to let the user select the reusable step to be added, if no set exists then user can add new (if he is authorized to do so) 
  • Followed by steps, will be the expected result (outcome) of the test procedure
  • Test case can be linked with one or more requirements raised in the "development project"
    For this a comma separated list of Ids of the requirements that need to be linked with the test case should be provided 
  • Custom fields that have been defined under the project that are to be used for test cases will be available for data entry / selection
  • Skip conditions, these will be available only when custom fields of type "Option box" and "Multiple Choice Option box" have been defined under the selected project that are to be used for test execution; skip conditions help the tester to avoid execution of unnecessary steps for a specific environment, and to avoid guess work
    • For this user can select a custom field to be used in condition and select the value for it to drive the condition
  • If required, user may upload file(s) to provide more relevant/detailed information to the tester
  • A number of tags can also be associated with a test case
    • For tags, user has an option of either selecting from pre defined tags or to define and add a new one

All the information collectively helps the tester to map out, how a part of a process should behave when executed.

Here Suite, Title, and Priority of test case are mandatory fields.

If at the time of adding the project "Separate text fields for steps and expected results" is enabled, then at the time of defining the test cases, instead of a single text field for all steps, separate text fields will be made available for entering individual test steps and their respective expected results, as shown in image below.


User has an option to add as many steps as he wants by clicking Add more steps link provided. User can also delete the steps using the Cross button. 

Reordering Steps

With version 1.16 of Test Collab comes the feature of ordering the steps. One can now drag and drop a step along with its expected result from one position to the other in steps list. 

 

Please note that the feature of reordering of steps would not work when HTML Editor is used as Editor Type for test case steps, expected result and description. 

Edit a Test Case

When an authorized user opts to edit a test case, then a form similar to add new test case will be shown and user can update any of the information he may wish to.

For user's convenience, the test case edit page now has navigation links to help switch to the previous or next test case in the sequence in which they have been placed on suites tree. 

Test Cases Versioning

As of July 2014, with the latest release, Test Collab now supports Test Case versioning, i.e. individual update in test case's data will be recorded separately as a revision.

User will have the option to see the change log wherein he can see the list of the changes a test case has been through since the test case was first created.

User can see an individual revision's details, also has an option to revert to a previous revision.

While editing editor has an option to provide notes pertaining to the changes he has done in the test case, for his or viewer's convenience.

 

Copy Test Cases

To avoid repetition of data entry efforts, as an option a user can create a copy of selected test case into the same suite, and use the case if little or no modifications are required.

User can copy a test case by simply selecting Copy option from the context menu shown when a test case selected on suites' dashboard, or by clicking on Duplicate icon on the test case details view pane.

 

 

Import Test Cases

This feature helps the user import test cases, from

  • A pre-exported CSV file, or
  • A custom created CSV file

This helps in creating reusable cases, and saves creation and management time for cases that have already been defined. 

The CSV to be used for importing the cases must follow a structure and some basic rules that are also related to test case creation. 

First of all make sure that the file you are using for import has the same syntax (Markdown / HTML / Plain text) as the one being used for your Test Collab's instance. 

Secondly check the size of file that is being uploaded, as the upload limit depends on the PHP settings done on your hosting server. 

The CSV can contain the data for following columns 

  • Title 
  • Description 
  • Steps 
  • Expected Result 
  • Priority

As they are all part of a test case defined in the application.

Off these, only Title and Priority are mandatory.

For Priority, the allowed values are 

  • Low
  • Normal, and
  • High 

To separate 2 steps under steps column leave a gap of a line between them.

In Test Collab, while creating a project you get an option to keep test case steps in single text field; or to have separate text fields for them so that expected results can also be maintained individually, two sample CSV files have been attached with document and these can be referred to create test cases that can then be imported into the application. 

When Import menu under Test Suites & Cases tab is used then an authorized user gets the options as shown below:

Opting to import the test cases from a CSV, a form like this will be shown:

 

 

On the form user will see: 

  • A test suite selector,
  • An uploader to upload the CSV (having cases to be imported for selected test suite), and
  • A box that can be checked if the first line of CSV represents headers, or defined fields in the table of cases being imported

A test suite selector lists the test suites that exist under the selected project, typing in the suites' selector box shows up the names of suites that match the criteria entered. 

As an option one can click on the "Select from tree" icon, this will show up the suites' list to select suites from the tree.

 

User will be required to select the suite for which he wants the cases to be imported. Empty suites will also appear in the list, to enable the user to import data for them.

Once suite is selected and file to upload has been chosen, user can proceed by clicking import button.

Next screen will allow the user to select the field used in Test Collab individually that maps to a CSV column (name shown as label for drop down).

When all mandatory fields (for a test case) have been mapped, then a process to merge the data from CSV into the test suite will start. If there are any issues while import then the same will be reported on screen. 

Please note that all the test cases present in CSV used will be imported under a single (selected) test suite. 

5.6. Reusable Steps

Reusable steps can be used to create a group of steps which are common among a lot of test cases as they are without any modifications.

Using reusable steps helps you in saving redundancy and time in test case creation process.

Please note that as of Test Collab version 1.14.2 , reusable steps can only be used when "Markdown" is selected as editor type in "General Settings"

Add a Set of Reusable Steps

When an authorized user opts to add a set of reusable steps, then a form will be shown to enter information.

 

On the form, user will be required to provide

  • Title of the set
  • Steps, these are the most important set of one or more instructions to execute the tests arranged in a logical sequence
    • In the editor provided for steps, he will have option to add the defined steps that are included under the test case,
    • For each step, user should be clear in defining the part of the procedure that step is expected to be used for,
    • To separate 2 steps user should provide a gap of 1 line by hitting Return (Enter) key twice,
    • To provide the properties (format) for text being used for each step, user will have set of icons associated with the text editor.

Both title and steps are mandatory fields.

Edit a Set of Reusable Steps

When an authorized user opts to edit a set of reusable steps, then a form similar to add new set will be shown and user can update any of the information.

Reusable Steps' List

The reusable steps' list screen, lets an authorized user manage the reusable steps.

 

On this screen, a button to add a new set of reusable steps, a list of pre-existing sets of reusable steps will be shown.

Using Reusable Steps 

While being on test case add or edit page, when a set of reusable step is created, user can embed it in test case by clicking the  icon on toolbar of text editor. When the icon is clicked user gets list of set of reusable steps to embed. User can also define new reusable steps from this screen. 

6. Test Executions Management

6.1. Test Plans Management

Test plan can be treated as a group of similar test executions with different configurations based on their properties (custom fields). You can create test plans on how , when and which test case will be executed and which team member will run it during a release cycle. 

Lets say you have 2 custom fields for your test execution:

  1. OS (option box type)
    • Linux
    • Mac
    • Windows
  2. Browser (option box type)
    • IE
    • Chrome
    • Firefox

Now if you want to run a set of your test cases against all these configurations, creating test executions for each configuration separately is cumbersome and doesn't show any relation between the created executions. That is when a test plan will comes in. 

With Test Plan, executions of different combinations of (m*n) are automatically created, and you should be able to select the configurations you want to run your test cases against without the repetitive steps.

So here is what you should do to create a test plan: 

    1. Opt to add a test plan 

    2. Opt to add an execution (group)



    3. Select the configuration fields first, since we are discussing browser testing on different OSes , so you would select custom fields: browser & os.



    4. Next step requires assignment of test cases to the testers



    5. Select the configuration combinations. If you selected all 3 OS and all 3 browsers, that would result in a 3 x 3 matrix i.e. 9 separate test executions would be created. 



    6. But before creation, it'll show all the combinations as such:
      1. OS: Linux, Browser: IE
      2. OS: Linux, Browser: Chrome
      3. OS: Linux, Browser: Firefox
      4. OS: Mac, Browser: IE
      5. OS: Mac, Browser: Chrome
      6. OS: Mac, Browser: Firefox
      7. OS: Windows, Browser: IE
      8. OS: Windows, Browser: Chrome
      9. OS: Windows, Browser: Firefox

        You can delete any particular item from the list. In above example, we know IE doesn't work in Linux and Mac. so we would use delete button next to these list items.



        Note:
         One can also add an execution irrelevant of above combinations to make it as a part of plan.

    7. After you have reviewed configuration combinations, you can continue with the same steps as followed for normal execution creation.
    8. Then this test plan will show up with test execution index, like any other test execution and with the metrics.

You also have flexibility to strategize your testing. Distribute each configuration level among different team members.

6.2. Test Executions Management

Test Executions are the entities that form basis for all major processes involved 

  • Assigning, 
  • Pre-Execution Review, 
  • Executing (and Bug Reporting in case of failures), 
  • Re-Executing (if needed), 
  • Reporting, and 
  • Collaborative Analysis of tests 

Here collaborative analysis include 

  • Analyzing, 
  • Discussing, 
  • Retesting, 
  • Reassigning, 
  • Overriding test result, 
  • Invite developers to comment on any executed test case 

Test Executions can be part of a test plan and can be related to milestones, making monitoring of milestones easier. 

Add a New Test Execution Record 

In Testcollab, adding a Test Execution is a 2 step process and it involves adding a new Test Execution, which is followed by the process of assigning the new Test Execution to a tester. 

User can reach to this screen by either using Test Execution ->Assign option or by using Allocate New Test Execution on Milestone's view page. If test execution is to be created under a test plan then "Add Execution" option can be used. 

When an authorized user opts to Assign a new Test Execution record, a form is made available to enter information.

On the form user will be required to provide: 

  • Title of the new Test Execution 
  • Priority, possible values for which are Low/Normal/High 
  • Relation the Test Execution can have with a pre-existing milestone (this is shown only when user has not been redirected to this page from Milestone's View page), and 
  • If some custom fields have been defined under the project, then the values for those could be set here 

Here custom fields can be seen as additional parameters that may be set to control the execution of a test. 

To know more about custom fields see this.

 

Here title is the only mandatory field.

Using Execution Templates for Defining Test Executions 

Test executions can now be created on the basis of execution templates

While creating a new test execution you have an option to "Use execution template" , when this option is checked you can select an existing execution template from drop down provided.

Once a template is selected you do not need to follow the second step of assignment which includes selection of test suites / test cases and also selection of assignee testers. Clicking "Next" would simply create a test execution on the basis of one or more test case selection criteria you have set while defining the selected execution template and the test cases would be assigned to tester(s) selected in execution template.

Assign Testers for a Test execution

This process helps an authorized user assign test cases to the testers collectively or individually, that are going to be the part of either a newly created Test Execution or of a pre-existing Test Execution.

Here user (assigner) is required to clearly define the relation between a tester and the test cases.

A tester can be a member in selected project having permission to execute tests, an administrator, or a remote executor defined under application settings.

User can assign one or more test cases that may belong to one or more test suites present in the selected project. 

Once the assignment is done then only it would be possible to get the tests executed (or atleast set their status). 

 

On the form:

  • Test suites present under the project will be available for selection (Clicking the test suite's name would expand the list of test cases defined under the selected test suite. User can now select test cases individually or collectively by checking their boxes).
  • Once the cases are selected, user can select one of the testers (from Assign To drop down) for the assignment.
  • After selection, one can see the testers' names next to the test suites' and/or test cases' names.

If there are some test cases for which testers have earlier been assigned, then for each such test case the name of assigned tester will be shown next to it in the test cases list box (until reassigned to someone else).

 

For convenience user can filter the test cases to be assigned, on the basis of their suite's title, test cases title, priority, linked tags and requirements.

When User has opted to filter the test suites and cases then he can simply key in the search criteria in the respective boxes he wants to filter the list on the basis of, once he starts keying the related options will be suggested and appropriate option can be selected.

Once the filter is applied, the list of test cases and their respective suites will be restricted to the criteria selected. When filter is applied, Reset option can be used to remove the applied filters and show all cases and suites.

After assigning, if email notification for assignment of a test has been enabled by administrator, then the same would be sent to assignees.

Edit a Test Execution

When an authorized user opts to edit a pre-existing Test Execution, then a form similar to add/assign Test Execution will be shown and user can update any of the information.

Test Executions' List

The Test Executions' list screen lets an authorized user manage the Test Executions.

 

User can see the list of existing Test Executions under selected project along with their relevant details. He gets options to perform various operations on Test Executions that he may be authorized for.

Then For each Test Execution details covered:

  • Time taken to get executed
  • Graphically represented test results – green indicates passed, red indicates failed, yellow indicates skipped, and grey indicates not yet executed
  • Its status, success
  • The member(s) it is assigned to 
  • Milestone it belongs to
  • Creation date, the person who had created it 

The links to perform various operations like View/Execute/Report/Edit/Assign/Delete are shown to the authorized users when hovered over the Test Execution details.

The list can be sorted on the basis of Test Execution

  • Title,
  • Priority,
  • Time taken,
  • The date the Test Execution record was created on, and
  • Its status

User can also set the filters for the list on basis of

  • Priority,
  • Status, and
  • Milestone
  • Test case title
  • Test suite title
  • Custom fields (if exist for test execution)

To filter, user can use Filter button provided (as shown in the image above).

Test Executions "Assigned To Me"

This screen particularly lists the Test Executions that have been assigned to the user who is currently logged in.

 

All options remain same as they are on Test Executions' list page, except for the fact that the list is limited to the tests that have been assigned to him enabling him to focus on those Test Executions and perform relevant operations including opting to execute them.

6.3. Executing Tests

Test Execution is an entity which groups together the test cases and their testers. They can be easily assigned and monitored.

Pre Execution

Before executing the assigned tests a tester goes through a pre-execution screen,

 

On this screen user will be shown summary of test(s) to be executed, which covers

  • Number of Test suites under the defined Test Execution
  • Number of test cases to be covered during the execution
  • Estimated execution time which depends on the previous executions for same tests
  • Name of the project member who has defined the Test Execution.

An option to either start or resume the execution depending upon the status of the previous executions will be available.

As parameter the priority and any other custom parameters (fields and their values) set for this Test Execution.

List of related test suites (the test cases belonging to which have to be executed)

For individual test suite covered in Test Execution, the details of cases under it covering individually test case name, and estimated execution time, will be shown on clicking the suite's title.

Running the Tests

After pre-execution screen on selection of either Start Now, or Resume option, test run screen is displayed.

If sequential execution / popup-window for execution is enabled then while user is executing/running the tests, a separate window to manage the running tests and provide his feedback on individual test case back to the Test Collab application, would be shown. 

Since this screen is shown on a separate small window it will not hinder the execution process for the test.

 

 

 

If sequential execution / popup-window for execution is disabled then the same window will be used to let the tester provide his feedback. This provides an option for tester to move back and forth between the executable test cases by directly selecting the same from suites tree. 

On the window the user can see

  • The progress of the entire test so far
  • The time so far spent during execution of a particular test case
  • Estimated time, if the test case has earlier been executed

User has the option to stop / resume the time logging for execution of this test case.

Followed by these will be details of the test case being executed, including

  • The test suite name
  • Test case title
  • Description
  • Steps to be followed, sequentially arranged in the order in which they are to be executed
  • Expected results for the test case, once all the steps are performed

For user to provide his feedback he has options to

  • Leave his comment
  • Upload one or more file to attach with the execution  record of the particular test case, the files thus attached, would also be linked with the issue at the time of reporting, in case of failure

Most importantly the user will be required to set the result of this test case; user can

  • Mark the test as passed, or
  • Mark the test as failed, or
  • Mark the test as failed and report the same through issue manager (if an issue manager has been associated with application and if the project specific settings have been defined for issue manager), or
  • He can simply skip the test       

Executions when separate text fields for steps and their expected results have been enabled for project.

If while creating the project you have enabled separate text fields for steps and expected results then the execution window will allow you to mark each step as passed / failed / skipped moreover tester can also leave comments for each step individually. 

If email notification for action(s) performed here has been enabled by administrator, then the same would be sent.

6.4. Test Executions View and Report

Viewing a Test Execution's Details

Once a Test Execution record has been created, the details of the same can be viewed on Test Execution details page.

One can see

  • Result - Even if one of the tests has failed, then also the Test Execution will be treated as failed
  • Success Factor Percentage - Ratio of number of test that passed to the total number of tests
  • Time Spent - the aggregate time spent on execution of all the tests

Graphically represented statistical information related ratio of passed / failed / skipped tests

Details like the priority and milestone under parameters

Execution Summary with number of test cases present in test execution, total cases executed, passed, failed, skipped, pending and their respective percentages. Number of defect reported (if any) with their IDs. 

List of Related Members – one, who created the Test Execution record and also the ones who have been assigned the tests.

Number of pending tests, if they are to be performed by logged-in user

A link to start the test case, either when the tests under defined Test Execution have not been executed at all, or there are some test that have been marked as pending.

If there are any unexecuted or pending tests, then user will see a Start Now button on the screen and when user opts to execute, then user will be redirected to a screen where user will have option to do the pre execution settings for the defined Test Execution and then execute it.

For individual test case covered in Test Execution, the details include

  • The member it is assigned to
  • Execution result
  • Time taken during execution
  • A link to view the execution details of individual test case

Bulk Actions

User, can perform actions like marking the selected test case(s) as Passed or Failed, setting the executed test case(s) for Retest, test cases can even be Reassigned to other tester by selecting the name from dropdown.In short the real team collaboration will be possible with this. Further Details.

If user selects to view individual test case, then he will be redirected to a screen where it would be possible to not only view the test case definition details, but also the results and logs of activities for the test.

Test Execution's Report

Once an execution has been completed i.e. all its tests have been executed, user can opt to view a report on the Test Execution.

All details shown will be similar to that of Test Execution's view screen.

The report contains result, success factor, and time spent for test execution. Graphical representation of test results along with parameters, members and executed test details.

Users have options of getting the report exported in PDF format, also the same report can be printed from being on report page only using the icons provided on top of the page.

6.5. Collaborate with Team

To let user collaborate with other team members over the details of a tests that are covered under a defined Test Execution.

Here a platform is offered to the testers, assigners, administrators and other project members to do the Collaborative Analysis of tests.

Here Collaborative Analysis include

  • Analyzing,
  • Discussing,
  • Retesting,
  • Reassigning,
  • Overriding test result,
  • Invite developers to comment on any executed test case

For this, user can either request to view details of a test case already covered under a Test Execution or he can perform the actions available on individual Test Execution view page.

 

This is a screen where it would be possible to not only view the test case definition details but also the results and logs of activities for the test.

User also has the option to Comment, Mark the Test as Passed, or Failed, or simply Re-assign the test to someone else. In short the real team collaboration will be possible with this.

On this screen, the information available includes

  • Result - Passed/Failed/Skipped/Not Executed
  • Time Taken - The time taken on execution of the test (if already executed)
  • The test case specific details
With the introduction of test case revisioning feature, one can see the exact revision of the test case that was executed during a particular execution run. The revision number is shown next to test case name under the test case specific details section. When user clicks on revision number, then he can see the test case's status at the time of execution.

Also available are links to navigate to other peer tests under the selected Test Execution.

Logs of activities by users (during either execution of the test case or during any operation available to user through this interface), which include

  • Operation performed, like marking the test as passed, or marking the test as failed, or skipping, or reassigning the test or simply commenting on the status
  • Comments left by user
  • The date on which the activity was performed (posted)
  • Reported Defect Ids with their links

If separate text fields for steps and expected results have been enabled for the project the execution case window will provide step wise execution results. 

Various operations can be performed by the user, depending on whether he is authorized for, he can

  • Comment on the status of the test,
  • Mark the test as passed,
  • Mark the test as failed, or
  • Reassign the test

When user opts to Comment (on) or Mark the test as Passed, or Mark the test as Failed, or Reassign the test case, then a separate dialog will popup, which will let the user select the action he wants to perform. Apart from that he can

  • Leave his comments
  • Attach file(s), if required 

 

 

 

On submitting the information pertaining to the action user has performed the log related to test case will be updated with the action the user has chosen along with his comments.

 

6.6. Execution Templates

Do you often have to define test executions that require similar test suites to be assigned to same group of testers? If yes, then you can now use Execution Templates which make the definition of test executions even more easier. 

An Execution Template can be treated as a basis for the test execution definition. Execution Templates once defined can be used to create new test executions on their basis, without any need to go through test case selection and assignment process for every test execution.

You will get the option to manage execution templates by navigating to Test Executions -> Execution Templates while working for a project. 

Adding an Execution Template

Adding an execution template is a straight forward process where you start by giving the template a name and then you provide test case selection criteria.There can be more than one test case selection criteria defined for a template.

For every test case selection criteria you either need to select the test suite that holds the test cases that you want to be included in template and eventually be part of the test execution, or you can also select tag if you want assign all test cases that have a particular tag associated with them. Please note that you can also have both test suite and tag selected for a particular criteria.

Once the criteria is defined you can select one or more testers who need to be assigned the test cases that fulfill the selected criteria. Please note that if more than one tester is selected for a criteria then the test cases would be assigned randomly.

There can be more than one sets of conditions defined under an execution template.

When an execution template is defined it can be viewed , edited or deleted.

Creating Test Executions on the Basis of Execution Templates

While creating a new test execution you have an option to "Use execution template" , when this option is checked you can select an existing execution template from drop down provided.

Once a template is selected you do not need to follow the second step of assignment which includes selection of test suites / test cases and also selection of assignee testers. Clicking "Next" would simply create a test execution on the basis of one or more test case selection criteria you have set while defining the selected execution template and the test cases would be assigned to tester(s) selected in execution template.

 

7. Reports

7.1. Execution Reports

Now move a step ahead in analyzing the test executions with the enhanced reporting.

Whether you need an overview on test executions or want to know how test cases with certain tags, requirements and priorities have fared, you can get all this information with graphical support. 

The report types included are :

  • Executions Summary Report, and
  • Coverage Report 

Executions Summary Report

When it comes to taking an overview of one or more test executions, Executions Summary Report comes handy. You simply need to select the test executions you want to have an overview of, and you will be presented with analytical and comprehensive report aided by metrics and graphs. 

To create an Executions Summary Report, you need to select one or more executions that you want to be covered for analysis.

 

 

Once a summary report has been created, you can view the same.

 
 

 

Coverage Report

 

To analyze how tags, requirements or priorities are covered under different test executions, you can get a coverage report generated.

A coverage report can be based on tags or requirements or priorities, and to create a coverage report, you need to specify those tag(s) / or / requirement(s) / or / priorities.

In the picture below a coverage report based on tags is being created.

 

After the report is created you can view the same.

 
 

Printing a Report

Every generated report can be printed, for this on the execution view page, you get a printer icon, clicking on the icon will open the print dialog for you, so that you can set the properties for the output you want to take.

7.2. Test Case Distribution Report

Analyzing the distribution of test cases becomes easier with Test Case Distribution report.

This report provides information on distribution of test cases on various criteria: 

Distribution by Test Suites

 

 Distribution by Tags

 

 Distribution by Priority

 Here is how the overall test case distribution report looks like.

 

8. Settings

8.1. General Settings

There are certain general settings that effect the behaviour of whole application and can be mananged from this screen

 

Settings include

  • Date format
  • Website time zone
  • Enabling of hint messages
  • Enabling test automation
  • Use of HTTPS for secured browsing
  • Editor type
  • Disable sequential execution / popup-window for execution
  • Attachments are private and only logged in users can access them

 

Setting Date Format for Application

Administrator can set the date format that can be used throughout the application.

Available options are

 

Format

Example

YYYY Month DD

2012 June 30

DD Month YYYY

30 June 2012

YYYY-MM-DD

2012-06-30

DD-MM-YYYY

30-06-2012

MM-DD-YYYY

06-30-2012

 

Once selected the date format will be applicable from that point onwards on forms and for any reporting, or communication through out the application.

Setting the Website Time Zone

Time zone can be set by selecting one from the options available.

 

Similarly Hint message can be enabled / disabled for the entire application.

 

Test Automation

It helps the tests in the application to be conducted automatically by configuring automation settings and also setting remote executors.

Use of HTTPS for secured browsing

Administrator can opt to use HTTPS for making the site browsing secured.

Before enabling, he can check whether the HTTPS is supported on his server, by using the link provided next to the checkbox.

If HTTPS is supported and administrator has opted to enable HTTPS, then immediately after saving these settings the site browsing will be through HTTPS only.

Any access to the site will then be SSL enabled, from that point onwards. 

Similarly, if user wants to not to use HTTPS in the future, then the same will be possible, by disabling the setting.

Editor Type

Some fields like steps and expected result show interactive text widgets. You can choose your default editor for all the projects.

The options that you have for the editor are: 

Markdown - The Default Editor 

Markdown is a lightweight markup language with plain text formatting syntax designed so that it can be converted to HTML and many other formats. When selected it provides you the editor that expects markdown syntax, and this is also the default editor for the application. 

HTML WYSIWYG Editor 

With this you get a What You See Is What You Get (WYSIWYG) editor so that you can instantly see how the text would look like. 

Plain Text 

When selected you will get a simple text box with no formatting options for the text added.

Disable sequential execution / popup-window for execution

When it comes to executing test cases there are two options of interface:

Sequential execution interface

This shows test cases to be executed one-by-one sequentially in a separate popup.

Non-sequential execution interface

This provides tree like structure of test cases with option to select the test case to be executed.

Attachments are private and only logged in users can access them

This option controls the privacy of attachments. In other words - this option, when checked restricts the accessibility of attachment for a person who is having a link for an attachment, but has not logged into the application.

For instance, while reporting a bug from application (using the linked issue manager), if a tester has attached a file, and added some comments, then the comments will be shown and the link for the attached file will be made available for the one who is reading the ticket created, now, if administrator has set the attachments as private, then that person would not be able to download that attachment unless he logs into the TestCollab application.

8.2. Email Settings

Test Collab provides a feature of automatically sending the emails to the related users whenever there is an activity performed during the usage of the application. 

Administrator can easily configure the related settings and enable or disable the notifications he may want to. 

Administrator also has the option to modify the template that will be used to compose the mails for its recipients. 

The templates are categorized / named to clarify the event/action they related to, administrator can, at any point for any template, choose to revert to the default template. 

The Form 

A form to let user define email settings 

 

The form will be divided in 2 sections 

  • One for templates been used by application for various actions, and 
  • The other for enabling/disabling notifications for various types of events 

Modifying a Template 

A form enabling the administrator to edit the template 

 

The form will be divided in sections, like section for 

  • Title of the mail, and 
  • Section for contents 

Here user can set contents for 

  • Email clients that support text only, and 
  • For the clients that support both text and HTML 

Each section allows user to change the text for that along with option to merge certain predefined fields that are relevant to the context (action) for which mail template will be used. 

And user can use the HTML tags inside the text box for HTML contents. 

Restore Default Template 

For any event, administrator can opt to restore the default template been offered by application. 

Enable/Disable Email Notifications 

When administrator opts to change settings for notifications, a form with the names of various types of actions for which email can be sent, will be shown 

 

8.3. Authentication Methods

Test Collab comes with the support for alternative authentication methods like LDAP that includes Active Directory services. 

To make use of LDAP integration in Test Collab, PHP's LDAP extension will be required to be installed on the server that hosts the application.

Configuring LDAP for Test Collab

Once installed, you can use LDAP as an alternate method of authentication for Test Collab.

Under the "Settings" tab of dashboard, you would find a setting with the name "LDAP"

 

 

On the LDAP configuration screen you would see the options to enable or disable LDAP Authentication for the application.

Option to Use default authentication as fallback, i.e, if default authentication is enabled, then the user who wants to login to the application would also be able to use default method of logging in apart from the LDAP authentication. It is recommended to keep this setting enabled.

Following entries are mandatory

  • Server Address - The address to access the server that runs the LDAP services you want to use
  • Server Port - In most of the cases it is 389
  • Bind DN - The unique identifier (the Distinguished Name) for the entry used for binding
  • Bind Credentials - consists of the credentials for bound entry to access the information (not mandatory)
  • User Context DN - The Distinguished Name that identifies the user context
  • LDAP User Search Filter - The filter string provided here will become basis for the search method that queries the LDAP database to look for user related information. 

LDAP User Search Filter explained :

This search filter requires the use of a specific string "[[username]]" (without quotes), as this string will be replaced by the string that the person willing to login to the application has provided for Username field.

Here a few sample search filters

uid=[[username]] - this would only search for the text entered for username in uid field

(|(uid=[[username]]) (mail=[[username]])) - this would look for text in both uid and mail fields that exist in LDAP database.

Testing the settings

Once the LDAP related configuration has been done, you can test the same by using the Test button available on the settings form.
 
 
On the form that opens to Test LDAP settings, you can enter "Ldap user id/name" and "Password". When you hit the "Verify" button after entering the username and password, connection to the LDAP server followed by validation of credentials entered. Any error either in connection or authentication would be reported to you.
 
When the settings are saved and tested, and if "LDAP Authentication" is enabled, you see following differences:
 
On user add/edit forms you will see a new field for providing "LDAP Username", so that it can be used for LDAP authentication at the time of logging in.
 
 
While logging into Test Collab, user would have to enter his LDAP authentication credentials, however if "Use default authentication as fallback" is set as an option in LDAP settings then user may use "Default Method" to login
 
 
 
 

9. Requirement Management

9.1. Requirements Management

Test Collab helps you organize your development project's requirements.

Irrespective of their sources they can all be managed and linked with test cases and can be analyzed through reports and traceability matrices. 

Please follow the links below to have details on how various sources of requirements can be managed and how traceability matrix can be generated.

 

9.2. In-built Requirements Management

Managing project requirements in Test Collab has become more easier as you can now have all your requirements maintained within the tool itself, without being dependent on your linked issue manager or any external tool, 

Introducing the In-Built Requirements Manager that helps you create, modify and remove Features and their Scenarios in which a feature's behaviour may change. 

To use this you need to select "Test Collab's In-built requirements manager" as the source while enabling requirements for your Test Collab project. 

Enabling Requirements Management for Project



Navigate to Requirements tab under Settings menu for the project, 

  • "Enable" the requirements management, 
  • Select "Test Collab's In-built Requirements Manager" as source of requirement 

Managing Features and Scenarios

Once requirements are enabled you can switch to "Requirements" tab, here you will get a link to manage the features and scenarios. 

On manage features page, you can use the relevant link to start adding the feature. 

Similarly scenarios can be maintained by using the "Scenarios" option available next to feature on features list page. In Test Collab you can have one or more features and in turn you can have one or more scenarios under each feature that would define how a feature is going to behave in a particular scenario. 

Linking Features and Scenarios with Test Cases

All features and scenarios once added can be linked with the test cases under the current project. 

See All Requirements under Current Project

All features and scenarios created under the current project are shown on requirements list page in hierarchical manner. 

 

On requirements index you can : 

  • Get the feature / scenario details viewed tool when you click on the requirement id,  
  • See the number of test cases linked with each requirement listed, 
  • Add test cases that link with the requirement, 
  • See the linked test cases on test suites and test cases management screen

Importing Requirements Through Excel or CSV Format Files

When in-built requirements are enabled for a project you can import the features and scenarios using Excel or CSV format files. 

The process for importing requirements through Excel or CSV is same for both features and scenarios. 

When on requirements index you see options to get the features imported into the project: 

On selecting either of the options you will be required to select the file that holds the requirements.

If the data file has headers i.e. the first line defines the header for data, then you can check the box available for the purpose. 

Once you select the file and click on Import, you will be redirected to a page where you can map the fields for requirements with columns in the file. Only two columns are mainly expected here - Title and Description 

After the mapping when you hit "Save", the process to import data from the selected file starts and if there is an error while importing the data then the same will be reported otherwise the requirement's data would be populated into your Test Collab project.

A sample Excel spreadsheet that holds some features has been attached with this article for the reference.

9.3. Linked Issue Manager as Source of Requirements Management

if your project's requirements are being maintained in the issue manager that is linked with your Test Collab instance, then also you can manage your requirements to perform some basic operations like :

  • Fetching the list of requirements from issue manager, 
  • Getting them linked with test cases, 
  • Get the list of all requirements linked in current project, 
  • Get the Requirements Traceability Matrix generated, 
  • Get the reports generated to analyze the requirements coverage

 

Navigate to Requirements tab under Settings menu for the project,

  • "Enable" the requirements management, 
  • Select "Linked Issue Manager" as source of requirement, 
  • If you want to refer to the requirements list from issue manager at the time of linking one or more requirements with test cases in Test Collab then type the "URL for Requirements List".

Linking Requirements with Test Cases

When you are adding a new test case or editing an exiting test case then you can provide one or more requirements separated by comma for a test case. and any number of test cases under the project can have requirements linked to them. 

See All Requirements Linked in Current Project

Once you have requirements linked with test cases then navigating to "Requirements" menu will show you all those requirements. 

On requirements index you can : 

  • See requirements list with their titles as maintained in linked issue manager, 
  • Get the requirement opened in the issue manager when you click on the requirement id,  
  • See the number of test cases linked with each requirement listed, 
  • Delete the requirement, 
  • See the linked test cases on test suites and test cases management screen  

 

 

9.4. External Tool as Source for Requirements

When the requirements for a project are neither being maintained in Test Collab nor in the issue manager that is integrated with your Test Collab instance, then you can get them managed by opting to keep the "External Tool" as the source of requirements for your project. 

Enabling Requirements Management for Project

Navigate to Requirements tab under Settings menu for the project, 

  • "Enable" the requirements management, 
  • Select "External Tool" as source of requirement, 
  • Type the "Link Pattern", the link pattern should be a valid URL that can be used to point to a particular requirement in the tool. Requirement's identifier would be represented by [[id]] because at the time of parsing this URL [[id]] will be replaced by the actual id that you provide to link with test case in your project.

Linking Requirements with Test Cases

You can provide one or more requirements separated by comma for a test case. and any number of test cases under the project can have requirements linked to them. 

 

See All Requirements Linked in Current Project

Once you have requirements linked with test cases then navigating to "Requirements" menu will show you all those requirements. 

On requirements index you can : 

  • Get the requirement opened in the external tool when you click on the requirement id,  
  • See the number of test cases linked with each requirement listed, 
  • Delete the requirement, 
  • See the linked test cases on test suites and test cases management screen 

 

 

9.5. Requirements Coverage Report

You have features and scenarios introduced as requirements in Test Collab and you want to know how well they are being covered in the project. 

Get the analytical coverage reports for requirements that belong to a specific version, or a specific module or to the entire project, by using the "Coverage Reports" option. 

 

When opted to view a report listed on 'Coverage Reports' page, you will be shown coverage details for the requirements that come under the criteria you selected, i.e. version / module / entire project; to let you know about the number of covered and uncovered requirements.  

Covered requirements here mean the requirements that have at least one or more linked test cases. 

Uncovered requirements refer to requirements with no linked test cases, therefore it helps you know about the test cases that should be created to cover all the requirements.  

Coverage % (shown graphically) is the ratio of covered requirements to the total number of requirements in your project/version/module. 

 

9.6. Requirements Traceability Matrix

Once you enable requirements and start managing them in Test Collab, then irrespective of their source you probably would like to know how requirements can be traced back collectively with the test cases they are linked to.

For this you can click on "Requirements" tab in navigation after selecting the project and then click "Traceability Matrix" link in sub-navigation.

 

 

Here an "X" mark indicates the relation between the requirement and its associated test cases.

10. Issue Manager Integration

10.1. Basic Issue Manager Integration Settings

Almost all popular bug tracking system being used currently can be integrated with the Testcollab.

The list includes, but not limited to:

It is also possible to integrate Testcollab with a custom issue manager.

All settings pertaining to the issues manager can only be configured by a user having administrative privileges.

Application Wide Settings

This feature lets the administrator define the application wide configurations for the issue manager association.

This will help the activities, especially any failure of a test being executed to be reported directly through the issue manager user uses.

Only one issue manager can be linked to an application at a time.

For configuring user should be aware of the URL used by issue manager, credentials - username and password or API key, and fields on the form been used by issue manager to report any new issue (if at all he wants to configure a few of them globally for the application).

The details for such field/custom field on "report" issue page, for which user wants a value to be set (and that is going to be static especially through out the application), can be gathered by exploring the source of the page for "Reporting a New Issue".

A form to let user define configuration for issue manager

 

On the form user will have option to

  • Select the issue manager (from a list) he wants to configure
  • Enter the Issue Manager's server address.
    Full URL should be put with http:// or https:// (in case of secured URL), it should end with a slash (/), Any file name after last slash of URL should not be added. As an example, it should be- https://defetcts.somedeomain.com/ not https://user.somedomain.com/secure/Dashboard.jspa 
    (Please note that, for Trac you have to include the project Id in the "URL of Issue Manager", so that the URL looks like http://<URL for Trac Instance>/<Project Id>)
  • User (used for logging into the issue manager) or API key, (as the case may be) for the specific issue manager, because bug trackers like JIRA and Fogbugz do not provide API keys.
  • And if username is entered then the password to log on to the issue manager should be provided

If user wants to keep certain fields that will be used while creating issues to have same values across all projects then user should define the same in the section available.

For this the user can enter name of the field/custom field on page used for issue reporting and the value that should be assigned to that particular field.

User may add as many fields as he wants for the purpose by using "Add more fields" option.

 

Testing the Settings

The settings can be tested before being saved.

When Administrator has opted to test the settings, a separate window will popup where in user will be shown the status of the connection to issue manager.

When Connection could be made, then a success message will be shown.

 

When Connection is not possible, then appropriate error message showing the cause of failure will be shown

 

Project Specific Settings

To enable issue manager integration for each project user must fill the project specific issue manager settings form.

To do so, user should go to dashboard of the project which he likes to integrate with issue manager.

A form to let user define project specific configuration for issue manager

 

On the form user will have option to select name of the field from the drop down, and the enter/select value that should be assigned to that particular field when a bug is reported.

You also now have an option to add a field that is not listed in the drop down provided, for this choose to "Add user-defined-field" from drop down, and enter the name of the field / custom field on page used for issue reporting.

Once you focus out of the text box provided to enter the name of user defined field, you get another text box to provide its value.

A filled up project specific issue manager settings form would look like this 

Testing the Settings

An option to test the configuration before saving the settings is available.

 

When Administrator has opted to test the settings for issue manager, a separate window will popup where in he will be required to confirm that he wants to test the configurations.

The title for new (test) issue will be shown. The title can be edited if required.

 

On confirming the operation by clicking the provided button, a new issue will be created; and the title, and the values for fields set during configuration along with their values will be shown.

A link to the newly created issue will be available so that the proper functioning can be verified.

Multiple User Integration

In TestCollab, it is possible for a user to link his issue manager account with his TestCollab profile, if he wants to be identified as a reporter for the bugs he report during the test executions. See Linking Profile with an Issue Manager for more details on this.

For further details on configuring various issue managers see this.

Tickets created under the selected issue manager can have links to the attachments that may have been added by tester at the time of test run. These attachments will always be available for the person who has also logged into the TestCollab application, at the time of using the link, BUT for the person who has not logged into TestCollab, the availability will depend on the accessibility option set by the administrator, under the application's General Settings options. See the related document for details.

10.2. Integrating Test Collab with Asana

To integrate Test Collab with Asana, you should have Asana Personal access token ready with you.

Here are the steps to get your Token generated, in case you do not know:

Being on Asana opt to goto "My Profile Settings...", You will be shown a popup similar to the one being shown below.

Under Apps tab click on "Manage Personal Apps". You will then be directed to "Developer App Management" page 

Create a new personal access token by clicking the given link. 

Enter the description of app you want the access token for. 

A new token will be generated and made available temporarily for copying. Make sure you get the new token copied as the same would not be made available again and in case you have not copied it then you would need to generate a new token. 

Configuring application wide integration settings for Asana in Test Collab:

 
Navigate to Main Dashboard > Settings > Issue Manager and provide the Personal access token (in place of username) after selecting Asana from the drop down available on the issue manager integration settings page.
 
 

You also have an option of allowing the reported issue to be resolved / closed automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Asana uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project wise setting for Asana, you will get the list of fields that are important for configuration. The list includes :
  • Workspace, and
  • Project
 
 
For these fields lists of options will automatically be fetched from Asana on the basis of credentials provided by you during configuring application wide settings.  
 
You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.

Once the values for all fields have been provided you can save and test the settings by creating a sample issue in Asana. 

What's Next

Once you have configured Asana as an issue manager , bugs can now be automatically reported into Asana if a test case fails while execution and the tester has opted to report. More information on this is available here

You can also get requirements from Asana linked with test cases in Test Collab so that you can have a better traceability and coverage analysis. For details on requirements management please refer relevant user guide article

 

10.3. Integrating Test Collab with Assembla

To integrate Test Collab with Assembla, you should have Assembla API key and API key secret ready with you.

Here are the steps to get your API key and API key secret, in case you do not know:

  • Access the profile page after logging into Assembla, 
  • Click on API applications & sessions
  • On the resultant page you will be able to see the API key and API key secret
 

Configuring application wide integration settings for Assembla in Test Collab:

 
Navigate to Main Dashboard > Settings > Issue Manager and provide the API key and API key secret (in place of password) after selecting Assembla from the drop down available on the issue manager integration settings page.
 
 

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Assembla uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project wise setting for Assembla, you will get the list of fields that are important for configuration. The list includes :
 
  • Project Id
  • Assignee Id
  • Priority
 
 
For fields like Project Id and Assignee Id lists of options will automatically be fetched from Assembla on the basis of credentials provided by you during configuring application wide settings. For Priority, you may provide any of the following values - 
  • Highest (1)
  • High (2)
  • Normal (3)
  • Low (4)
  • Lowest (5)
 
 
 

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.

A completed project specific issue manager setting form for Assembla would look like this.
 
 
Once the values for all fields have been provided you can save and test the settings by creating a sample issue in Assembla.

10.4. Integrating Test Collab with Bugzilla

Almost all popular bug tracking system being used currently can be integrated with the Testcollab. 

Make sure you have selected the issue manager, for this go to  Settings->Issue Manager tab from the account dashboard, where you can define application wide settings for issue managers. More details.

Once done you can select the project for which you want to do the project specific settings.

This page explains how integration with Bugzilla is possible in Testcollab.

Important Note

To integrate Bugzilla with Test Collab, all required fields of Bugzilla's new bug form must be added in project's issue manager settings page. For example, if you have created a custom field named priority which is a required field then you must put field name and value for that field in issue manager settings page of project, else Bugzilla won't allow Test collab to submit any bug.
 

A default Bugzilla settings form has 6 pre-entered fields-

product - In Bugzilla bugs are categorized into Products. You have to write your product name in which you want all reported defects to be posted. Use exactly same as it is specified in your Bugzilla.

component - Components are second level category and each belongs to a product. You have to put name of component of the product you have mentioned above in which you want all defects to be posted. This is a required field.

assigned_to - Here you can specify the email address of the user to whom you want to assign all defects created from Test Collab. This field is an optional field, if you will leave this field empty all bugs will be assign to default assignee of that component.

version - Fill the version name to which all the defects of the particular project belongs. For example - 1.0, 1.2 etc. This is a required field.

op_sys - Fill the Operating system name which should be selected while creating bug in Bugzilla. For example- Linux, Windows XP etc. This is a required field.

rep_platform - Fill the hardware platform name which should be selected while creating bug in Bugzilla. For example- All, PC etc. This is a required field.

You can click the test button at bottom to make sure that Test Collab is able to create a sample bug in Bugzilla.

Setting up custom Bugzilla Fields

1. Go to Bugzilla's Enter Bug page.

2. Right click on it and select View source.

3. Find the name and value for Bugzilla's custom fields. Name for bugzilla custom starts from cf_

4. Click on add more field link on Issue Manager Settings page, put your custom field name and value in respective fields.

5. Click Test button to create a sample bug and check if your custom field was added successfully.

10.5. Integrating Test Collab with FogBugz

Configuring application wide integration settings for FogBugz in Test Collab:

While defining the application wide issue manager settings you need to provide the URL for the Fogbugz instance you want to associate, along with the email and password as part of the access credentials.

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Fogbugz uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings 

When it comes to project specific settings, values for following fields can be provided for Fogbugz, please note that all these fields are optional:

Project - Fill your project ID here. Use exactly same name as it is specified in your FogBugz.

Area - In Fogbugz each project is divided into functional areas. Use exactly same name as it is specified in your FogBugz like Code, Documentation, User Interface etc..

Category - Here you can specify that the issue you are reporting is a bug, feature or a question. Use exactly same name as it is specified in your FogBugz like Bug, Feature, etc..

Assigned to user id - Provide the name / id of the user to whom you want to assign all defects created from Test Collab. 

Priority - You can define the priority of the case here. You can provide the code for priority in value field

Tags - If you want your reported cases to have any tag then you can simply write your tags in the value field. You can provide more than one tags by separating them with a comma

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the  name of the field.

A completed project specific issue manager setting form for Fogbugz would look like this.

Once the values for all fields have been provided you can save and test the settings by creating a sample issue in FogBugz.

10.6. Integrating Test Collab with Github

Configuring application wide integration settings for GitHub in Test Collab:

While providing "URL for issue manager" make sure you enter https://api.github.com instead of https://github.com followed by username (or email) and password and fields (if required to be provided global values).

 

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Github uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project specific settings, values for following fields can be provided:

  • Repository name - the repository to which you want the bugs to be reported in case of a test failure 
  • Github name of owner of repository - GitHub username
  • Assignee - User who should be assigned the automatically created bugs
  • Milestone
  • Labels
 
 

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the  name of the field.

A completed project specific issue manager setting form for GitHub would look like this.
 
 
For "Labels" field you may provide value from these 
 
 
Once the values for all fields have been provided you can save and test the settings by creating a sample issue in GitHub.

10.7. Integrating Test Collab with Jira

Configuring application wide integration settings for JIRA in Test Collab : 

To integrate JIRA with Test Collab, you should have the URL, username and password ready that you use to access your JIRA instance. 

On your Test Collab instance use "Issue Manager" option under "Settings" menu on main dashboard. 

Here is the form that you get while defining the global issue manager settings: 

 

Since the integration helps you in automatically reporting the bugs into JIRA when a test cases fails during execution and the tester has opted to report the failure; there is also an option on the global settings page that allows to automatically set the status of reported issue as resolved or closed, when the related test case passes in Test Collab in subsequent test runs. 

This can be done by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that JIRA uses for closed or resolved issues, you need to enter the same for "Status code for resolved".  

You can test the connection with your JIRA instance to check whether the settings would work or not. 

When clicked on "Test" you would see a window popped up to show the status of the connection. 

 

 

Project wise integration settings 

To configure project specific settings for the integrated issue manager, switch to the project and use "Issue Manager" option under "Settings" menu. 

On project specific issue manager settings page, values for following fields can be provided for JIRA: 

 

project - Code of JIRA project under which you want the issues to be reported need to be entered here.

issue type - Specify the type of issue that you want to be created when a test case fails during execution. 1 for Bug, 2 for New feature, and so on

assignee - Here you can specify the username of the user to whom you want to assign all issues created from Test Collab. This field is an optional field, if you will leave this field empty all bugs will be assigned to default assignee of that project.

priority - You can define the priority of the issue here. Go to add issue page, and click on view source and you will see the value for priority in similar format- <option value="2" style="background-image: url(/images/icons/priority_critical.gif);" class="imagebacked">Critical</option>. So, value for critical is 2. Similarly default value for blocker is 1, major is 3, minor 4 & trivial 5.

components - You can define components here. Go to add issue page, and click on view source and you will see the value for components in similar format- <option value="10000" title="Addons ">Addons</option> So, in this case value for our component addon is 10000.

affectsVersions - You can define version in which the issue is manifesting here. Go to add issue page, and click on view source and you will see the value for versions in similar format- <option value="10000">1.0</option> So, in this case value for our affected version 1.0 is 10000.

fixVersions - You can define version to which this issue belongs here. Go to add issue page, and click on view source and you will see the value for versions in similar format- <option value="10000">1.0</option> So, in this case value for our fix version 1.0 is 10000. 

Adding other fields 

You can also add other fields as may be required to make the integration complete including any custom fields using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field and its desired value. 

A completed settings form

A completed project specific issue manager setting form for JIRA would look like this.

After the values for all fields have been provided you can save and use the "Test" button to test the settings by creating a test issue in JIRA.  

When "Test" button is used a window pops up and asks for the title you want to keep for the test issue

In the next step the status related to test issue creation is shown 

 

What's Next

Once you have configured JIRA as an issue manager , bugs can now be automatically reported into JIRA if a test case fails while execution and the tester has opted to report. More information on this is available here

You can also get requirements from JIRA linked with test cases in Test Collab so that you can have a better traceability and coverage analysis. For details on requirements management please refer relevant user guide article

Bi-directional integration is also possible allowing you to perform all your test case management related tasks from JIRA without any need to login to Test Collab. Please refer this article for details.

 

10.8. Integrating Test Collab with JIRA REST

This issue manager works with JIRA 5 and above, and internally uses REST API instead of deprecated SOAP API.

Rest of the working is same as JIRA issue manager. 

For details on integration with JIRA please refer - http://support.testcollab.com/index.php?pg=kb.page&id=8


10.9. Integrating Test Collab with Lighthouse

This page explains how integration with Lighthouse is possible in Testcollab.

Configuring application wide integration settings for Lighthouse in Test Collab:

While defining the global settings provide the URL, along with Username and Password used to access your Lighthouse instance..

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Lighthouse uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project specific settings, values for following fields can be provided for Lighthouse:

Fields are briefly described here:

project_id (required) : This is the Id of project and can be fetched from URL of ticket creation for example in xyz.lighthouseapp.com/projects/79931-some-project/tickets/new, Here, 79931 is the project id.

assigned_user_id : To get id of user to whom ticket should be assigned, open page source of create ticket page and search for "ticket_assigned_user_id" see the value attribute of options tags to fetch the assigned_user_id

milestone_id : This is the Id of milestone to which ticket can be assigned. Open page source of create ticket page and search for "ticket_milestone_id", value attribute options tags which are next to select box with id attribute ticket_milestone_id are id's of milestone.

state : This is the state of ticket. Enter the name as they appear in select box on ticket create page.

tag : This field is for tags that can be associated with the ticket. Enter a comma separated list for this. 

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the  name of the field.

A completed project specific issue manager setting form for Lighthouse would look like this.

You can click the Test button at bottom to make sure that Test Collab is able to create a sample bug in Lighthouse.

10.10. Integrating Test Collab with Mantis BT

This page explains how integration with MantisBT is possible in Testcollab.

Global Settings

While defining the global settings provide the URL, along with Username and Password used to access your Mantis instance.

 

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Mantis uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project specific settings, values for following fields can be provided for Mantis:

project_id (required) : Id of project for which issue is being created.To get project_id open /bug_report_page.php page and view page source.


search for "project_id" in page source, value attributes of option tag which follows select tag having name attribute equal to "project_id" are id of projects.

category_id (required) : Id of category to which issue should belong. Search for select box with attribute name equal to "category_id", value attribute of options tag of this
select tag are id's of category which can be used here.

reproducibility : Similarly like above fields search for select with name attribute equal to "reproducibility" and select values from the options tag.

severity : Search for select with name attribute equal to "severity" and select values from the options tag.
 
priority : Search for select with name attribute equal to "priority" and select values from the options tag.

handler_id : User id to whom issue should get assigned. Search for select with name attribute equal to "handler_id" and select values from the options tag.

view_state : Search for input tag with name equal to "view_state" and enter value attribute information of the view_state.

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.

A completed project specific issue manager setting form for Mantis would look like this.

Once the values for all fields have been provided you can save and test the settings by creating a sample issue in JIRA.

10.11. Integrating Test Collab with Pivotal Tracker

To integrate Test Collab with Pivotal Tracker, you should have Pivotal Tracker API key ready with you.

Here are the steps to get your API key, in case you do not know:

  • Access the profile page after logging into Pivotal Tracker, 
  • On the profile page scroll to the bottom, here you should see your API TOKEN 
 

Configuring application wide integration settings for Pivotal Tracker in Test Collab:

 
Navigate to Main Dashboard > Settings > Issue Manager and provide the API key (in place of password) after selecting Pivotal Tracker from the drop down available on the issue manager integration settings page.
 
 
You can test the connection to the Pivotal Tracker instance by using the "Test" button.

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Pivotal Tracker uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project wise settings for Pivotal Tracker, you will get the list of fields that are important for configuration. The list includes :
 
  • Project Id
  • Current State
 
 
For Project Id list of options will automatically be fetched from Pivotal Tracker on the basis of credentials provided by you during configuring application wide settings. For "Current State", you may provide any of the following values - 
  • accepted
  • delivered
  • finished
  • started
  • rejected
  • planned
  • unstarted
  • unscheduled
 
You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.
 
 
A completed project specific issue manager setting form for Pivotal Tracker would look like this.
 
 
Once the values for all fields have been provided you can save and test the settings by creating a sample issue in Pivotal Tracker.

10.12. Integrating Test Collab with Redmine

This page explains how integration with Redmine is possible in Testcollab.

Important Note

To integrate Redmine with Test Collab, you have to fill your Redmine's Api key in issue manager settings page and you can leave the password field empty.

And, you must have "REST web services" enabled in Redmine to get API access key. This can be done by logging in as an administrator, and on Administration page, select Settings then Authentication, and check "".

Obtaining Redmine API key.

redmine_api_access_key.png

1. Login to your Redmine account

2. Go to My account page

3. In Right column under API access key, click the show link.

Configuring application wide integration settings for Redmine in Test Collab:

Here is the issue manager integration global settings form:

 

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Redmine uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project specific settings, values for following fields can be provided for Redmine:

 

Fields are briefly described here:

project_id - You have to specify your project identifier here. Project Identifier is displayed on settings page of each project on information tab. It is a required Field.

priority_id - You can define the priority of the issue here. Go to add issue page, and click on view source and you will see the value for priorities in similar format- <option value="1">Highest</option>, so our value for highest priority is 1. Similarly value for high is 2, for normal is 3 & for low is 4.

assigned_to_id - Here you can specify the value of the user to whom you want to assign all issues created from Test Collab. To obtain value for a user, click on view source, and you will see the value for assignee in similar format- <option value="10">Shriti Gupta</option>, so our value for user Shriti is 10. You can similarly find values for other users.

status_id - You can set the status of your tickets from this field. To obtain value of a particular status, click on view source, and you will see the value for status in similar format- <option value="2">Reopened</option>, so our value for user reopened status is 2.

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the  name of the field.

A completed project specific issue manager setting form for Redmine would look like this.

You can click the Test button at bottom to make sure that Test Collab is able to create a sample bug in Redmine.

10.13. Integrating Test Collab with Team Foundation Server

To integrate Test Collab with Team Foundation Server (TFS), you should have Team Foundation Server Personal access token ready with you.

Here are the steps to get your Token, in case you do not know:

Being on TFS opt to edit your profile, You will be shown a page similar to the one being shown below.

Under Security tab you would see the option to manage Personal access tokens , here "Add" option can be used to get a new token generated for use.

Enter the Description and select values for "Expires in" make sure that you select a longer expiry to avoid frequent regeneration of token.  Also select the scopes to match your needs again please make sure that you provide sufficient privileges so that issues can be reported and set as resolved / closed from Test Collab. When you click the "Create Token" button, a new token will be generated and made available temporarily for copying. Make sure you get the new token copied as the same would not be made available again and in case you have not copied it then you would need to generate a new token.

 

Configuring application wide integration settings for Team Foundation Server in Test Collab:

 
Navigate to Main Dashboard > Settings > Issue Manager and provide the Personal access token (in place of username) after selecting Team Foundation Server from the drop down available on the issue manager integration settings page.
 
 

You also have an option of allowing the reported issue to be resolved / closed automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that TFS uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project wise setting for Team foundation server, you will get the list of fields that are important for configuration. The list includes :
  • Project
  • Work Item Type Name
 
 
For these fields lists of options will automatically be fetched from TFS on the basis of credentials provided by you during configuring application wide settings.  
 
You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.

A completed project specific issue manager setting form for TFS would look like this.
 
 
Once the values for all fields have been provided you can save and test the settings by creating a sample issue in Team foundation server. 

What's Next

Once you have configured Team foundation server as an issue manager , bugs can now be automatically reported into Team foundation server if a test case fails while execution and the tester has opted to report. More information on this is available here

You can also get requirements from Team foundation server linked with test cases in Test Collab so that you can have a better traceability and coverage analysis. For details on requirements management please refer relevant user guide article

10.14. Integrating Test Collab with Trac

This page explains how integration with Trac is possible in Testcollab.

Configuring application wide integration settings for Trac in Test Collab:

While defining the global settings provide the URL, along with Username and Password used to access your Trac instance..

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Trac uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project specific settings, values for following fields can be provided for Trac:

 

Fields are briefly described here:

project - You have to specify your project name here. 

issue type - Here you can specify that the issue type you are reporting is a defect, enhancement or anything else. You have to write name of type in your value field of Test Collab, for example, for defects write defects in value.

priority - You can define the priority of the issue here. You have to write name of priority in your value field of Test Collab, for example, if you want tickets to have priority set as minor then write minor in value field.

milestone - Here you can specify the milestone for all defects created from Test Collab. You have to write name of milestone in your value field of Test Collab.

component - Fill the Component name to which all defects created from Test Collab should be assigned. You have to write name of component in your value field of Test Collab.

version - Fill the version name to which all the defects of the particular project belongs. For example - 1.0, 1.2 etc.

reporter - Here you can specify the username of the user who is reporting issues from Test Collab. This field is an optional field, if you will leave this field empty all issues will be set as reported by Project Leader.

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the  name of the field.

A completed project specific issue manager setting form for Trac would look like this.

You can click the Test button at bottom to make sure that Test Collab is able to create a sample bug in Trac.

10.15. Integrating Test Collab with Trello

To integrate Test Collab with Trello, you should have Trello API key and Permanent token ready with you.

Here are the steps to get your API key, in case you do not know:

After logging into Trello and then visit https://trello.com/app-key, You will be taken to a page that would display Developer API keys and secret.

For now we are only going to use the Key in making a call to request a permanent (authorization) token, and the request should be like: 

https://trello.com/1/authorize?key=[yourApplicationKey]&scope=read%2Cwrite&name=[myApplication]&expiration=never&response_type=token

Make sure that you replace [yourApplicationKey] with the Key you have copied earlier and [myApplication] with the name for your application to provide access to. An important part of the request is to ask for expiration = never and scope = read,write. After the request is made you will need to allow the application to use your Trello account.

 

Once you allow application you will be shown the permanent access token.  

You need to to take a note of the application key as well the newly generated token.

Configuring application wide integration settings for Trello in Test Collab:

Navigate to Main Dashboard > Settings > Issue Manager and provide the API key (in place of username) and Permanent token (in place of password) after selecting Trello from the drop down available on the issue manager integration settings page. 
 
 

For URL of Issue Manager you can use https://api.trello.com/1 .

You also have an option of allowing the reported issue to be resolved / closed automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Trello uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project wise setting for Trello, you will get the list of fields that are important for configuration. The list includes :
 
  • Board
  • List
 
 
For these fields lists of options will automatically be fetched from Trello on the basis of credentials provided by you during configuring application wide settings.  
 
You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.

A completed project specific issue manager setting form for Trello would look like this.
 
 
Once the values for all fields have been provided you can save and test the settings by creating a sample issue in Trello. 

What's Next

Once you have configured Trello as an issue manager , bugs can now be automatically reported into Trello if a test case fails while execution and the tester has opted to report. More information on this is available here

You can also get requirements from Trello linked with test cases in Test Collab so that you can have a better traceability and coverage analysis. For details on requirements management please refer relevant user guide article

10.16. Integrating Test Collab with Unfuddle

This page explains how integration with Unfuddle is possible in Testcollab.

Configuring application wide integration settings for Unfuddle in Test Collab:

While defining the global settings provide the URL, along with Username and Password used to access your Unfuddle instance..

You also have an option of allowing the reported issue to be resolved automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that Unfuddle uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project specific settings, values for following fields can be provided for Unfuddle:

Fields are briefly described here:

project-id - You have to specify your project id here. Project Id can be taken from project's URL. For example, if your project's url is http://gpo.unfuddle.com/a#/projects/18747 then your project id is 18747. This is a required field.

priority - You can define the priority of the ticket here. Go to add ticket page, and click on view source and you will see the value for priorities in similar format- <option value="5">Highest</option>, so our value for highest priority is 5. Similarly our value for high is 4, for normal is 3, for low is 2 & for lowest is 1. This is a required field.

assignee-id - Here you can specify the value of the user to whom you want to assign all tickets created from Test Collab. To obtain value of a user, click on view source, and you will see the value for user in similar format- <option value="11027">Shriti G.</option>, so our value for user Shriti is 11027. You can similarly find values for other users.

status - You can set the status of your tickets from this field. You have to write full name for status in your value field of Test Collab, as an example new, reassigned, resolved etc.

You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the  name of the field.

A completed project specific issue manager setting form for Unfuddle would look like this.

You can click the Test button at bottom to make sure that Test Collab is able to create a sample bug in Unfuddle.

10.17. Integrating Test Collab with YouTrack

Configuring application wide integration settings for YouTrack in Test Collab:

 
On your Test Collab instance, navigate to Main Dashboard > Settings > Issue Manager and provide the username (or email) and password you use to login to your YouTrack instance, after selecting YouTrack from the drop down available on the issue manager integration settings page.
 
 

You also have an option of allowing the reported issue to be resolved / closed automatically when the related test case passes by checking "Resolve reported issues when a failed test case is marked as passed". When you check this you will be prompted to provide the status code that YouTrack uses for resolved issues, you need to enter the same for "Status code for resolved".

Project wise integration settings

When it comes to project wise setting for YouTrack, you mainly have to set the value for Project field and that is important for configuration. 
 
 
For Project field , the list of values will automatically be fetched from YouTrack on the basis of credentials provided by you during configuring application wide settings.  
 
You can also add other fields including any custom field using "Add user defined field" from the drop down, clicking this you will get a text box in place of drop down to enter the name of the field.

Once the values for all fields have been provided you can save and test the settings by creating a sample issue in YouTrack. 

What's Next

Once you have configured YouTrack as an issue manager , bugs can now be automatically reported into YouTrack if a test case fails while execution and the tester has opted to report. More information on this is available here

You can also get requirements from YouTrack linked with test cases in Test Collab so that you can have a better traceability and coverage analysis. For details on requirements management please refer relevant user guide article

 

10.18. Integrating Test Collab with custom Issue Manager

Though Test Collab supports most of the major Issue Manager systems available in the market. But still if you don't find your Issue Manager in our list, with basic knowledge of PHP you integrate Test Collab with your issue manager.

Structure of IssueManager Module: All files of issue manager are located in /app/issue_managers. As pretty evident from this folder, there is one class for each issue manager, with name of file like <issue_manager_name>_issue_manager.php and name of Class like <IssueManagerName>IssueManager. Also all issue manager classes inherit class AbstractIssueManager which is in the file abstract_issue_manager.php. Any class with this structure will be automatically recognized by the system and will be shown in Issue managers list.

 

Step by Step Procedure to create your Issue Manager class:

  1. Lets say you want to  report issues to a issue manager called as Test, then create a file called as test_issue_manager.php in /app/issue_managers and a class with name of TestIssueManager inherting AbstractIssueManager.
  2. Create a static method getFieldProperties which should return array of fields.
  3. Create  createDefect method with three parameters title,description and attachment.
     This is the main method which is used to report defect and contains the all logic to call your issue manager and report the issue.
  4. If you want to send attachments added at time of reporting bug along with the issue then add logic to send attachments with issue also.
  5. If issue is created then return Instance DefectInfo containing information about issue created else throw IssueCreateException.
  6. Create checkConnection method to check connection with your issue manager server.

NOTE:

This is a basic starting guide to Integrate your Issue Manager. All the technical details regarding method parameters, properties are specified in comments of abstract_issue_manager.php. So you must go through all comments before integrating, also you can look at integration code of other issue manager classes.

AbstractIssueManager::$apiServerUrl, AbstractIssueManager::$username, AbstractIssueManager::$password can be used to call your server and for authentication.

There is no restriction that only issue managers with xml API can be used.

All issue manager classes which are packaged with this software, use logic depending upon their issue manager API's.

For Example :

JiraIssueManager - uses Soap and nu_soap classes.

UnfuddleIssueManager, RedmineIssueManager, FogbugzIssueManager  -  uses xml

LighthouseIssueManager and MantisIssueManager - Don't use any API rather replicate browser type behaviour by using  Pear HttpRequest Class.

TracIssueManager and BugzillaIssueManager - Uses xml rpc and xml_rpc classes.

11. Two-way Integration

11.1. Introduction

The term two-way integration or bi-directional integration here relates to the use of issue managers to perform test case management operations on Test Collab without requiring you to login to Test Collab application. 

Supported operations that can be performed from within the issue manager, include :

  • Operations possible when you use a TestCollab administrator's API key

    • Linking existing TestCollab project with current issue manager's Project

    • Creating new TestCollab project being in issue manager to link it with current project

    • Linking Existing test cases defined in TestCollab to an issue defined in issue manager

  • Operations available for all Test Collab users, provided their respective roles permit them to do so:

    • Adding a new test case and get it automatically linked with the current issue defined in issue manager

    • Editing an already linked test case

    • Viewing a test case details

    • Unlinking a linked test case from the current issue

    • Deleting a linked test case

    • Assign test cases to users who have permissions to execute test cases in TestCollab

To download the plugin for your issue manager, please visit plugin download page.

Please note that two-way integration is supported only on Test Collab version 1.3.4 and higher.

11.2. Two-way Integration with JIRA (OnDemand)

Add-on Installation

You can easily add Test Collab add-on to your JIRA OnDemand instance by following these steps : 

  • When logged into JIRA as an administrator, select "Administration" option 
  • Use "Add-ons" option under the "Administration" menu 
  • Key in "Test Collab" to "Search the Marketplace" textbox 
 
  • Clicking on Install button you would be asked to confirm the installation 

  • Installation should then begin

  • Once installation is done you will be shown a confirmation message
  • Installed add-on can then be configured
  • For "Test Collab URL", you simply require to enter the URL that is to be used to access Test Collab application for bidirectional integration from JIRA

 

Setting up Test Collab API key

 

To set up key to make calls to Test Collab API, one needs to follow these steps:

  • Access "Profile" of currently logged in user
  • Select "Test Collab Profile"
  • Enter Test Collab user API Key in the box provided. 

How to know Test Collab user's API Key

Being logged into Test Collab, navigate to your profile. On profile page, you will find your API key, that can be used to access Test Collab.

 

 

 

After setting up Test Collab add-on and user API key, you can start performing test case management related tasks.

If you are logged in as an administrator of JIRA application and also you have provided the API key for a Test Collab user who can act as an administrator, then you can enable "Test Collab" integration for a JIRA project and use an existing Test Collab project, for this switch to project settings page and select "Test Collab" option. 

 

Non Administrative Tasks

Please note that all the tasks that have been described below can be performed on the basis of role that has been assigned to the user whose API key is used, for the Test Collab project linked with the current JIRA project.

Once Test Collab is enabled for a JIRA project, you will be able to perform all other test case management tasks. For this you can either use an existing JIRA issue or create a new one.

Once an issue is created or selected, you will see a new section (tab) with title "Test Cases" below issue details that will allow you to manage the linked Test Collab test cases, this section gives you option to either add new test case in Test Collab and link it automatically to JIRA issue, or link existing test cases.

If you are a Test Collab user who is assigned a role that permits to edit test cases in a project, then you can link existing testcases under that Test Collab project.

When you click on "Link Existing", a popup would come up and that will list all test suites that are present in linked Test Collab project.

You can expand individual test suite to select specific test cases  

If you have selected the entire suite(s), all their related test cases will automatically get linked with JIRA issue.Otherwise individually selected test cases will be linked.

If you opt to add a new test case then you will get a popup to enter the details of new test case, the interface would be similar to the add page you see in Test Collab.

After saving, the new test case will be introduced in Test Collab application and will be automatically linked with the JIRA issue.

When one or more cases are linked with the JIRA issue, for each linked case you will get options like edit, unlink and delete. 

Opting to edit will popup the related form. Unlink will remove the test case from the list of linked test cases.

Deleting apart from unlinking from JIRA, will delete the test case from Test Collab instance as well.

Assigning Test Cases to Testers for Execution

Opting to "Create Execution", after selecting one or more linked test cases, will popup a window to first let you create a new Test Collab test execution and then assign the execution to the users who have the role in Test Collab project that allows them to execute a test.

Creating a New Test Execution 

 

 

 

Please note that two-way integration is supported only on Test Collab version 1.3.4 and higher.

11.3. Two-way Integration with JIRA (Hosted)

Add-on Installation

If you have a running instance of JIRA on your server, you can easily add Test Collab add-on to it. 

To download the compressed Test Collab add-on files for JIRA goto Downloads page. 

Once you have the add-on's .jar archive, you can get it installed in JIRA. 

  • When logged into JIRA as an administrator, select "Administration" option
  • Use "Add-ons" option under the "Administration" menu
  • Select "Manage Add-ons". You may be asked to reenter your password before moving ahead
  • On "Manage Add-ons" page, you will get a link to "Upload add-on".
 
  • Select the downloaded Test Collab add-on's archive and click "Upload"
  • This should also start installation once file is uploaded
 
  • Once installation is done you will be shown a confirmation message
  • Installed add-on can then be configured
  • For "Test Collab URL", you simply require to enter the URL that is to be used to access Test Collab application for bidirectional integration from JIRA

 

Setting up Test Collab API key

 

To set up key to make calls to Test Collab API, one needs to follow these steps:

  • Select "Administration" option
  • Click on "User Management" on top menu to see a drop down having related options 
  • Select "Users", to see the list of users that have been added into JIRA
  • Hover over username, on popped up box, click on "More"
  • Select "Administer User"
  • Select "Edit Properties"
  • In "Add User Property" section provide a new key with the name "TCKEY" and the value that would be same as your Test Collab's API key, you may like to refer "How to know Test Collab user's API Key" for details.

 

 

How to know Test Collab user's API Key

Being logged into Test Collab, navigate to your profile. On profile page, you will find your API key, that can be used to access Test Collab.

 

 

Performing Test Case Management tasks

After setting up Test Collab plug-in and API key, you can start performing test case management related tasks.

If you are logged in as an administrator of JIRA application and also you have provided the API key for a Test Collab user who can act as an administrator, then you can either add a new JIRA project or use an existing project to "Enable" Test Collab for it, for this switch to "Components" option on project administration page.

 

Adding a Project in Test Collab

As an administrator, you also have an option to introduce a new project in Test Collab, that will have the same name as that of the JIRA Project. For this you can select "Create New" from the drop down next to "Linked Project".

Non Administrative Tasks

 

Please note that all the tasks that have been described below can be performed on the basis of role that has been assigned to the user whose API key is used, for the Test Collab project linked with the current JIRA project.

Once Test Collab is enabled for a JIRA project, you will be able to perform all other test case management tasks. For this you can either use an existing JIRA issue or create a new one.

Once an issue is created or selected, you will see a new section at the bottom that will allow you to manage the linked Test Collab test cases, this section gives you option to either add new test case in Test Collab and link it automatically to JIRA issue, or link existing test cases.

If you are a Test Collab user who is assigned a role that permits to edit test cases in a project, then you can link existing testcases under that Test Collab project.

When you click on "Link Existing", a popup would come up and that will list all test suites that are present in linked Test Collab project.

You can expand individual test suite to select specific test cases

 

If you have selected the entire suite(s), all their related test cases will automatically get linked with JIRA issue.Otherwise individually selected test cases will be linked.

If you opt to add a new test case then you will get a popup to enter the details of new test case, the interface would be similar to the add page you see in Test Collab.

After saving, the new test case will be introduced in Test Collab application and will be automatically linked with the JIRA issue.

When one or more cases are linked with the JIRA issue, for each linked case you will get options like edit, unlink and delete.

Opting to edit will popup the related form.

Unlink will remove the test case from the list of linked test cases.

Deleting apart from unlinking from JIRA, will delete the test case from Test Collab as well.

Assigning Test Cases to Testers for Execution

Opting to "Execute Selected Tests", after selecting one or more linked test cases, will popup a window to first let you create a new Test Collab test execution and then assign the execution to the users who have the role in Test Collab project that allows them to execute a test.

Creating a New Test Execution 

 

 

 

Please note that two-way integration is supported only on Test Collab version 1.3.4 and higher.

11.4. Two-way Integration with Redmine

Plugin Installation 

If you have a running instance of Redmine ready with you and you have access to its files, you can easily add Test Collab two-way Integration plug-in to it. 

Compressed Test Collab two-way Integration plug-in files for Redmine can be downloaded from Downloads page. 

1. Once you have the plugin archive, then depending on the version of Redmine being used, you can extract the plug-in into : 

For Redmine 1.x - /apps/redmine/htdocs/vendor/plugins

For Redmine 2 and above - /apps/redmine/htdocs/plugins 

2. Restart Redmine. 

Make sure your Redmine's hostname and path are correctly configured to the same URL which you type in your browser to access it. 

Setting Host name and path 

For this :

  • Navigate to Administration page
  • Select "Settings"
  • On "General" tab of Settings page
  • Set the value of "Host name and path" equal to the URL you use to access your Redmine application followed by /redmine (Please note that the URL should not be prefixed with http:// or https://)

 

For example, if you access your Redmine application at redmine.abc.com
then the value for "Host name and path" should be like redmine.abc.com/redmine

Configuring Test Collab plugin

Next is to setup the Test Collab plugin, for this :

  • Navigate to Administration page
  • Select "Plug-ins"
  • Click on "Configure" option for the plug-in with the name "Test Collab"
 
  • For "Enable Test Collab? " , select "Yes"
  • For "Test Collab URL", enter the URL that is to be used to access Test Collab application for bidirectional integration from Redmine 

Non Administrative Tasks

The tasks that can be performed by a Redmine user, who need not be an application administrator

Setting up Test Collab API key

To set up key to make calls to Test Collab API, select "My Account" from top menu.

On "My Account" page, enter the API key for the Test Collab application user whose credentials you want to use to perform test case management tasks on Test Collab from Redmine, you may like to refer "How to know Test Collab user's API Key".

 

 

How to know Test Collab user's API Key

Being logged into Test Collab, navigate to your profile. On profile page, you will find your API key, that can be used to access Test Collab.

 

Performing Test Case Management tasks

After setting up Test Collab plug-in and API key, you can start performing test case management related tasks.

If you are logged in as an administrator of Redmine application and if you have provided the API key for a Test Collab user who can act as an administrator, then you can either add a new Redmine project or use an existing project to "Enable" Test Collab for it.

Adding a Project in Test Collab

As an administrator, you also have an option to introduce a new project in Test Collab, that will have the same name as that of the Redmine Project. For this you can select "Create New" from the drop down next to "Linked Project".

Please note that all the tasks that have been described below can be performed on the basis of role that has been assigned to the user whose API key is used, for the Test Collab project linked into the current Redmine project.

Once Test Collab is enabled for a Redmine project, you will be able to perform all other test case management tasks. For this you can either use an existing Redmine issue or create a new one.

Once an issue is created or selected, you will see a new section at the bottom that will allow you to manage the linked Test Collab test cases, this section gives you option to either add new test case in Test Collab and link it automatically to Redmine issue, or link existing test cases.

If you are a Test Collab user who is assigned a role that permits to edit test cases in a project, then you can link existing testcases under that Test Collab project.

When you click on "Link Existing", a popup would come up and that will list all test suites that are present in linked Test Collab project.

You can expand individual test suite to select specific test cases

 

If you have selected the entire suite(s), all their related test cases will automatically get linked with Redmine issue.Otherwise individually selected test cases will be linked.

If you opt to add a new test case then you will get a popup to enter the details of new test case, the interface would be similar to the add page you see in Test Collab.

After saving, the new test case will be introduced in Test Collab application and will be automatically linked with the Redmine issue.

When one or more cases are linked with the Redmine issue, for each linked case you will get options like edit, unlink and delete.

Opting to edit will popup the related form.

Unlink will remove the test case from the list of linked test cases.

Deleting apart from unlinking from Redmine, will delete the test case from Test Collab as well.

Assigning Test Cases to Testers for Execution

Opting to "Execute Selected Tests", after selecting one or more linked test cases, will popup a window to first let you create a new Test Collab test execution and then assign the execution to the users who have the role in Test Collab project that allows them to execute a test.

Creating a New Test Execution 

 

 

 

Please note that two-way integration is supported only on Test Collab version 1.3.4 and higher.

Known Bugs

Test Collab user's API key can only be updated from My account page, this is not possible from user add or edit forms.

12. Test Automation

12.1. Test Automation with Remote Executors V2

A remote executor is a machine that is used to execute your automated tests. For connecting the machine with your TestCollab setup, you will need to install remote executor on your machine. Currently we support Windows and Debian based Linux systems.

Prerequisites:

  • Test Automation setting is enabled. The setting can be enabled from "General Configuration" page by an admin 



  • Your Testcollab installation should be accessible from machine where remote executor is installed. If you are using self hosted instance on private network machine then TestCollab should be accessible on same network. For Cloud (SaaS) accounts, machine having remote executor installed should be able to access internet.

Configuring TestCollab Setup For Remote Machine:

To add a machine with remote executor to TestCollab. You can do it From Settings -> Remote Executors.

Adding remote executor will need following information:

  • Machine name: This is the name which will represent machine in the application.
  • Machine API Key: This can be any random series of alphabets that will be used for authentication of remote executor. This field should be unique for each remote executor. You can also opt to get an API key automatically generated by the system. 

    Please note that the length of the API key you provide should not exceed 15 characters

 

Installing on Windows:

Remote executor can be downloaded from this page.

  1. After you run the downloaded package, you will be prompted to enter two parameters. Your Test Collab Host and Remote executor API key.




  2. Next you will be required to confirm the path where remote executor files would be copied 



  3. In Test Collab Host, add the URL of your TestCollab installation and in Remote executor API key, add the unique API key that you entered while adding remote executor on TestCollab.
  4. Similarly follow the rest of the wizard - and you will see the success message if everything goes fine.

 

Note: All configuration changes will require a service restart. For Windows, Go to 'Run > services.msc > Right click Test Collab Remote Executor > Select Restart'.

Installing on Linux:

Remote executor can be downloaded from this page. If you are using a 64 bit Linux the please use 64 bit release.

  1. Change to the directory you downloaded the package into.
  2. Give executable permissions to the file using the following command :
    chmod 755 testcollab_remote_executor_lin64.sh
  3. Run installer file as root using either of these commands, depending on your platform 
    sudo ./testcollab_remote_executor_lin32.sh

    OR

    sudo ./testcollab_remote_executor_lin64.sh



  4. Follow the wizard as it installs. During installation you will be prompted to confirm the path where tcmre would be copied 



  5. Next would be the URL you use to access your Test Collab instance 



     
  6. And finally the unique API key that you entered while adding remote executor on TestCollab.

 

Note:

  • If at the time of installation you see a daemon related error "Job for tcmre.service failed because the control process exited with error code" , then you may need to add
    #!/bin/sh
    at the beginning of /etc/init.d/tcmre

  • All configuration changes will require a service restart. Linux users can start remote executor using the command command 'sudo /etc/init.d/tcmre start' or 'sudo service tcmre start'. You may replace 'start' with 'stop' or 'restart'. For Windows, Go to 'Run > services.msc > Right click Test Collab Remote Executor > Select Restart'.

Installing on Mac:

Remote executor is attached with this guide and is ready to be downloaded : 

  1. Extract the downloaded archive (tcmre_for_Mac.zip) into /opt/share/ 

  2. Edit /opt/share/tcmre/config.ini with your settings i.e. Machine API Key and Test Collab server URL 

  3. Test the tool by running the following command in terminal: 
    /opt/share/tcmre/bin/node  /opt/share/tcmre/src/index 

  4. If you want the remote executor to get launched automatically when your system starts, you can copy /opt/share/tcmre/tcmre.job.plist in /Library/LaunchDaemons to set it as daemon.

  5. Restart your system 

Please note it is important to configure your firewall or router settings if they do not block applications to listen on specific ports.

For further clarification, please refer - http://testcollab.com/files/automation/tcm_controller.swf 

12.2. Configuring a project and its test cases for automation

Project wise Test Automation Settings

This is the second step that needs to be followed to configure test automation for your Test Collab project, the first step that requires configuring the remote executor is explained here

Navigate to "Settings" > "Automation Settings" when a project is selected. 

On "Test Automation Settings" page the information under following heads need to be provided: 

  • Base Dir 

    The directory that would serve as base directory from where the automation tool needs to be invoked. This can be treated as the working directory for test automation tool. 

  • Command template to execute individual test case 

    The template that includes actual command to invoke the automation tool along with its parameters. The parameters need to be separated by white space and must be enclosed within a set of two square brackets.  

  • Command Parameters 

    The list of command parameters in a sequence in which they should appear with command. This list will be used while automating individual test case. 

  • File(s) to collect after test execution 

    These files will automatically be attached to your executed test case. So you should specify the absolute path from where the file(s) need to be fetched. 

 

Automating Individual Test Case

Once the automation settings for a project are in place then you would see "Automate" option for the test cases that come under the project.  

 When "Automate" option is selected, you will be given an option to provide test case wise parameters that should be passed to the command (refer template above) that invokes test automation tool. Individual parameter will have its own entry box to provide the value for. 

Having done with the configuration and automating individual test case, you can now easily assign the test cases to the "Remote Executor" machine so that it can in turn invoke your test automation tool with command and its parameters that are included while configuring. 

More details on test execution assignment are available here 

For further clarification, please refer - http://testcollab.com/files/automation/tcm_controller.swf 

12.3. Running Parameterized automated test cases

When you have deployed your application on multiple servers you might want to reuse same test scripts to test multiple environments. You can easily do that with our test automation feature. To demonstrate, we'll assume that you have a web application deployed on a staging and a production server.

Now during a typical application life cycle you might run into issues which requires you to verify both of the environments individually.

We'll do just that using parameter-based automation commands.

We'll assume that you have already specified automation related commands and details in your test cases. If not, please check it out here.

Defining execution-specific variable

Another requirement is that you should have custom properties defined for the executions. For web app that is to be tested on multiple servers, we'll define 'URL' as custom variable (or custom fields) for executions.

You can do so by going to your project, then Settings > Custom Fields

Running parameterized automated tests

Now you'll be prompted with additional text box asking "URL" every time you create an execution.

For creating this execution I've specified my URL which is now ready to be passed on to the machine that will carry out the actual execution.

Before that, let's see how we can insert variables like URL in our automation command template. Go to Settings > Automation Settings

 

On this page we'll write our command that triggers our automated test cases. Note that we have to parameterize it too so individual test cases can be executed.

We're using our dummy command as stated above:

testrun [[TestCase]] [[TestSuite]] --url [[Exec.url]]

The first two parameters are test case specific, the last one [[Exec.url]] is our execution-specific variable.

[[Exec.url]] will automatically be replaced by the URL you specify at the time of creating the execution.

Other user cases & ideas

Similarly, you can have a mobile app to test with multiple mobile platforms. In that case you would need to create 'platform' named custom field and so on.

Cross browser or cross OS testing.

12.4. Common Automation problems

Problem: My automation script works fine when triggered from command line but fails when triggered by Test Collab.

Solution:

Such errors can be annoying and end up taking a lot of your time. So if it has happened to you, we hope these steps can be of help:

Test Collab Remote Executor, the tool which triggers your automation script runs as a service in background. With Windows automation scripts/macros it is a common occurrence that services running in background usually cannot access objects/windows as we see on frontend. So make sure you verify that your scripts aren't failing because of this issue.

To do so, we'll simple first stop the Test Collab background service and then run the daemon manually. 

1. Stop Test Collab remote executor service:

Type services.msc in 'Run' > Find service named "Test Collab Remote Executor", right click and then Stop.

2. Now run the daemon in frontend manually:

Go to Run > Type 'cmd' and change current directory to path where you installed Test Collab Remote Executor

(Default path is: C:\Program Files\Test Collab\Remote Executor)

cd bin
node ..\src\index

This will show messages like:

checking for new execution to execute
Request info from http://localhost:8000/index.php/r...
Setting to check again in few minutes

3. Now trigger your automation script from Test Collab web panel or API once more and see how it goes.

If it succeeds, disable your service and create a simple batch file like below (just to give you an idea) in startup folder that runs Remote executor daemon in frontend.

cd C:\Program Files\Test Collab\Remote Executor\bin
node ..\src\index

 

12.5. Running GUI based tests on Windows

TestCollab Remote Executor is installed as a service by default on Windows. This prevents GUI based tests like AutoIt scripts, and desktop application testing to be executed from Test Collab Remote Executor due to Windows Session 0 security measure. 

As an alternative to the service. Remote Executor can be added in windows startup programs list, which will give Remote Executor the permission to execute GUI based tests. 

Following steps are needed to implement this after Remote Executor installation: 

  1. Disable the Remote Executor Service: For this run removeService.bat present in Remote executor installation directory, generally it is "C:\Program Files\Test Collab\Remote Executor\bin" 
  2. Copy Startup batch file to startup directory: Copy or create shortcut of "C:\Program Files\Test Collab\Remote Executor\bin\remote_executor_startup.bat" to your system startup directory. For 64 bit directory use remote_executor_startup_64.bat file 
  3. You may also need to add a scheduled task to launch remote executor on system startup 
  4. Reboot the system 

Additional Notes: 

  • For 64 bit system Remote Executor is installed at "C:\Program Files (x86)\Test Collab\Remote Executor\" 
  • Startup directory is generally located in "C:\Documents and Settings\<user>\Start Menu\Programs\Startup" or at "C:\Users\<username>\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup" depending upon the Windows version. Also you can run "shell:startup"  command using run console (Shortcut keys combination: Win+R) 
  • If you are using unattended windows VM or machines. Then please make sure either automatic login is enabled for user or you can login into the Windows to launch the desktop. 

12.6. Test Automation with Remote Executors V1

 

NOTE: This document is out of date now. We have released new version of Remote executor. Please follow instructions on for new version given here: http://support.testcollab.com/index.php?pg=kb.page&id=165

 

A remote executor is a machine which is used to execute your automated tests. You can have any number of remote executors in your account. You can install a new remote executor with these simple steps:

Download the remote executor package from this link.

For Windows Machines:

1. After you run the downloaded package, you will be prompted to enter two parameters. Port to listen on & Your Test Collab Host.

3. You can find out your Test Collab Host in the e-mail sent to you at the time of signup or you can go to our home page and press login button. Then fill in your e-mail address and you'll be taken to your Test Collab URL. Please note it is important you do not add trailing slash ('/') at the end of URL.

4. Similarly follow rest of the wizard - and you will see the success message if everything went fine.

For Linux Machines:

1. Change directory to the directory you downloaded the package.

2. Give executable permissions to the file with:
chmod 755 install.sh

3. Run install.sh file as root
sudo ./install.sh

4. Follow the wizard as it installs the remote executor as service and uncompresses files.

5. Edit /opt/tcmre/config.ini (optional)

Note: All configuration changes will require a service restart. For linux users, run this command: 'sudo service tcmre restart'. For Windows, Go to 'Run > services.msc > Right click Test Collab Remote Executor > Select Restart'.

Verify if executor daemon is running

Open http://127.0.0.1:1337/0123456789/ping in your browser. If you see 'TCRESuccess' message, it means everything is running normally.

Security Settings

API / Security key should be edited after installation for best results. Be sure to reflect this API key on Test Collab server if remote executor is already added.

Adding Remote Executor in your Test Collab account

Note down the IP address / hostname and port number of this machine, you will need to specify these details on your Test Collab panel. Log in to your Test Collab panel, then go to Settings > Remote Executors to add this machine.

 

Please note it is important to configure your firewall or router settings if they do not block applications to listen on specific ports.

For further clarification, please refer - http://testcollab.com/files/automation/tcm_controller.swf

13. Miscellaneous

13.1. Automatic Screenshot Upload Feature

While testing a software you often need to capture screenshots and make the same available as part of your testing record. Generally you would take a screenshot save it in a file and then upload the same as an attachment, this seems straight forward but in the routine when you have to do this for virtually every case you are testing, then this becomes tedious. At Test Collab we have come up with an easier solution that requires you to just use Print Screen key on your keyboard when you want the screen currently being viewed to be captured and the same to be attached automatically with the test case you are executing on your Test Collab instance.

A tool called Shrex is required to push the captured screenshots to Test Collab. It is an open source tool available at https://getsharex.com , download the tool and install it on the PC that you use for testing. To configure Sharex you need a custom uploader script like the one given below:

{
"Name": "Test Collab",
"DestinationType": "ImageUploader",
"RequestType": "POST",
"RequestURL": "https://mytestdomain.app.testcollab.com/v2/screenshots/upload?api_key=TestCollabAPIKey",
"FileFormName": "file",
"ResponseType": "Headers"
}

Please note that in the script above you need to replace https://mytestdomain.app.testcollab.com with your Test Collab instance URL and TestCollabAPIKey with the Test Collab user's API key. Here is how you can get your API key:  

Being logged into Test Collab, navigate to your profile. On profile page, you will find your API key, that can be used to access Test Collab.

 

Once you have changed the text, you can save the script in a file with the name tc.sxcu please note that you need .sxcu as extension so that it can be recognized by Sharex.

Double click the saved file, Sharex will ask for a confirmation before adding the script as custom uploader. 

Click Yes

We now need to set Test Collab (the name given in the script) as default destinations for the screenshots captured via Sharex. For this in Sharex click on Destinations and select Image Uploader Custom Image Uploader > Custom Image Uploader.

 

Another setting that is needed is for After Capture Tasks.

Make sure that Copy image to clipboard, Save image to file and Upload image to host are all selected. 

After these settings have been done for Sharex, you are now all set to get screenshots automatically uploaded while running a test case. 

When a test case execution run window is opened in Test Collab and you press Print Screen key on keyboard the screenshot will be automatically taken, saved into a file and would be uploaded with the test case being run. 

You should see the thumbnail under Screenshots section on test case run screen. 

These screenshots are available every time you view the executed case details. 

If you want you can also annotate the screenshots for this you need to add another task of Annotate Image under After capture tasks.

With this task added whenever you would execute a test you can first annotate the image being captured.

 

We have also prepared a video to demonstrate the use of automatic screenshot upload utility and it is available at https://www.youtube.com/watch?v=lR0TM10dMOs 

13.2. Working with multiple branches of an application

For many teams it is essential to work with different branches at same time so the main repository stays stable while development can still progress at a fast rate. Developers can create their own branches from the trunk/baseline and work independently on it.

This creates a few problems when it comes to testing:

1. With the changes that occur in branch, some test cases are affected too and needs to be changed.

2. Multiple branches means maintaining multiple copies of a test case.

3. Testing individual branches requires a tester to refer to the updated list of test cases only.

4. Some branches will be merged to trunk/master sooner than later, that means at some point you will be merging these updated test cases to the trunk/master.

In this article we'll describe how to solve these problems step-by-step.

Maintaining multiple copies and changes of test cases

To get started, you should have all your test cases in the trunk created in Test Collab as shown below. We'll take a sample project as example here to make things clear.

 Then create a new suite named 'Branches' where you'll store different branches your team is working on.

Now a branch will affect several test cases as it is actively developed. Say, our branch 'My first branch' affects all test cases in 'Clients' suite in different ways.

To manage this, you can simply copy whole test suite 'Clients' under 'My first branch'.

Now you have a while copied suite 'Clients' under 'My first branch'. This can be done for individual test cases as well. Also you can create more than one branches and have multiple copies of test case and suites.

After this is done, you can now proceed to make changes in your test cases independently in different branches. A change in test case or addition of a test case will live only under than branch and is completely independent of baseline or other branches.

Testing Individual Branches

After you are done with managing the structure and copies of test cases in different branches, you'll eventually need to execute these.

You'll need to: a) test trunk/master, b) test individual branches

Testing trunk is simple:

To test a different branch, you'll need to select the suite under the branch name instead of first-level suite. In this case, 'Clients' suite.

Rest of the test execution process will follow. More details here

Merging the test case changes after branch is merged

Currently there is no way of automating the merge function and this must be done manually. We recommend opening the test case in multiple windows and compare them side-by-side.

Once you figure out the changes to make. You will need to update the test case (Edit link on test case view) at the baseline as indicated below:

After changes are done, you can see the updated test case with change log:

You can also run a side-by-side diff to see what was changed.

 

13.3. Managing Versions and Modules

To better manage your development project you can get the versions and various modules introduced into the system. 

To start with navigate to 'General' tab under the 'Settings' menu. here you will get the options to manage versions and modules for current project.

Versions Management

Manage Versions page allows you to :

  • See the list of versions already introduced,
  • Add new versions,
  • Edit or delete existing versions.

 

Adding a new version requires entering the title for it.

Similar screen is made available when you opt to edit a version.

Modules Management

Manage Modules page allows you to :

  • See the list of modules already introduced,
  • Add new modules,
  • Edit or delete existing modules.

 Adding a new module requires entering the title for it.

 Similar screen is made available when you opt to edit a module.

Once you have versions and modules introduced they can be linked with the features present in the project. The option of linking a version and a module is provided at the time of creating new feature or modifying an existing feature.

13.4. Some issues that are not really the issues in Test Collab

Issue: I do not see option to link requirements with my test cases

Reason: Requirements feature need to be enabled by the user who has administrative privileges, the option is available under project wise settings menu for administrators.

Issue: I do not see "Fail & Report Bug" button while executing test cases

Reason: The "Fail & Report Bug" option appears only when issue manager settings have been configured globally as well as for the current project. Details on issue manager configuration are available here - http://support.testcollab.com/index.php?pg=kb.chapter&id=8

Issue: I clicked on "Fail & Report Bug" button while executing a test and no defect was shown to have been reported when I was redirected to test execution view

Reason: The defects are not reported immediately as this is taken care of by a background task and there is a frequency set for its execution, if you are using a self hosted instance then you can ask your server administrator about the frequency set for cron job (on Linux like environments) or background task (on Windows like environments); on SaaS server the frequency is 10 minutes. 

Issue: When I use "Fail & Report Bug" option, the issue gets reported but I am not being shown as reporter of the defect!

Reason: You may not have linked your issue manager account with your Test Collab user account, to do so you can edit your profile and use the option "Link my <IM> Account" here <IM> represents the name of the issue manager configured for your Test Collab instance like JIRA, Redmine, TFS etc.

Issue: A user has been added onto my Test Collab instance but I am not able to assign test cases to that user

Reason: The user must not have been added as a team member for the current project, or the role assigned to user must not be having "Execute Tests" permission included. More details on roles management are available here- http://support.testcollab.com/index.php?pg=kb.page&id=38

Issue: I want to report failures of test cases from one Test Collab project into multiple projects of linked issue manager 

Reason: You can do so by having a custom field added in Test Collab that would hold the names of projects from issue manager and then map the custom field in Test Collab to the field from issue manager that represents its projects. More on custom fields and fields mapping is covered here - http://support.testcollab.com/index.php?pg=kb.page&id=101 and http://support.testcollab.com/index.php?pg=kb.page&id=112

Issue: I do not see the left pane on test suites and test cases dashboard!

Reason: A key press must have been causing this, Ctrl + Left Arrow keys combination can be used to make the left pane on test suites and test cases dashboard appear.

13.5. Bookmarking pages

Bookmark the Pages

You can now bookmark a page in testcollab and send the link to anyone, when the user requests to open the page, if he has not logged-in then he will be required to go authentication process, and once authenticated he will automatically be redirected to the requested page.

13.6. Keeping Separate Text Fields for Steps vs Single Field

Because of popular demand, with the launch of version 1.14.1 on June 25th 2016, we have added a new feature that takes defining test case steps, their expected results and their execution to a detailed level. 

While adding a new project on your Test Collab instance you will get option to enable "Separate text fields for steps and expected result".

When separate text fields for steps is not enabled then you enter all the steps in a single text field one after the other by separating them using two consecutive new line characters, and to provide expected results you again have single box. 

If you opt to enable separate text fields for steps and expected results then at the time of creating test cases you will get an interface that would allow you to define steps in separate text fields and also provide expected result for each step individually. 

Let's look at how execution window looks when you have not opted to use separate text field for steps and expected results : 

If you notice steps are clubbed which is fine for majority of projects or testing styles. However some projects might require more granular control over execution steps, to keep a track at which step a particular case failed.

Let's look at how this new feature gives you more flexibility:

Now there a pros and cons to both the approaches and you should choose the style that suits more to your team. If you don't know - just go with the flow and leave this setting.

Advantages of using Separate Steps:

1. Granular control over execution so your team knows exactly where the case was failed.

2. Comment on each step separately.

Disadvantages of using Separate Steps:

1. Requires more time in writing test cases.

2. Requires more time while executing test cases.

 

Test Case Management
Test Collab Support
Product Tour