For every software developer testing is like an ABC of programming. This is due to the fact that without proper testing when the application is rolled out to the general public the development team will find lots of bugs, which will not only hinder the availability of the service but will create serious security risks for anything using said software.
What is Testing?
Testing is the process of evaluation of a developed system and all of its components to make sure that it satisfies all of the requirements. In short, executing the system to identify bugs, errors, exploits, etc. This is done via analyzing the software item to detect the difference between what you expect the program to do and what it actually does.
Who does the testing?
This depends on the actual process as well as the stakeholders of the project. In the IT industry, usually bigger companies have specific teams dedicated to evaluating the developed software. The developers themselves also conduct their own tests called Unit Testing. Most of the time, every member of the team tests parts of the project by their own expertise. These can be roughly divided into four groups:
- Software Testers
- Software Developers
- Project Manager
- End-User
These may be done by Software testers, Software Quality Assurance Engineers, QA analysts, etc.
When does testing start?
The testing should start as soon as it is possible. This is mainly due to minimizing the costs after the development process ends. Correcting bigger flaws once the software is fully finished will take a much bigger amount of time and resources than it would have if the testing was done during the development process. A good example of this would be video games. For example, Playerunknown’s battlegrounds have faced a lot of issues with the optimization of the game. Frame rate drops are very frequent even on the beefiest of the machines. It is mainly due to the fact that the game was built on the engine without proper testing as the team was not as knowledgeable about the inner works of the engine. There are myriads of small details to consider and it is not easy to do so when developing such a large scale game. Now that it has rolled out, to fix all of the required issues may even require a total rebuilding of the whole software. Now it all depends on how much money Bluehole wants to put into their game and how acceptable is the condition for the end-user to continue bringing profit.
This is a more or less relaxed case scenario since if the game at hand is, per se, casino related like slots as casinos are turning more towards online gaming as they see it as a better and more comfortable bet. However, the improper testing may result in malicious users having the ability to tinker with the odds to turn an unfair profit. Due to this, financial losses may be at hand as well as some security exploits that may be used to manipulate the system into bringing the odds higher for specific malicious users. Usually, real money casino slots services are going through rigorous testing phases to make sure no such issue arises. However, one of the biggest threats to a company is a security vulnerability, which can be used to get into the player database and steal a lot of personal information including names, personal identification numbers, credit card information, etc. resulting in endless lawsuits, angry customers, and effective shutdown of the whole operation. Statistically, this is one of the most expensive fields in the industry as the security costs are off the roof. However, as more casinos are moving towards the online environment they become more invested in security as well with Microsoft investing over $1 billion a year only in the software security sector.
Because of such threats the testing is done at almost all stages of the development including but not limited to:
- During requirement gathering, where analyzing and verifying the requirements is considered a part of testing.
- During the design phase via reviewing the design to improve the system
- On the completion of the code from the developer
When does testing end?
It is hard to determine when the testing should end as it is almost impossible to develop a piece of software with 100% of its vulnerabilities and bugs fixed. The end-user will always make or break the software at one point or another. Even if the software’s integrity has been top-notch for years on straight some of the day 1 exploits can still be present in the system, which just has not been found yet.
However, there are some waypoints:
- Testing deadlines
- Once the test case execution is complete
- Once the functional code is completed
- Once code coverage happens at certain points
- When no high priority bugs are identified
- By management’s decision
Quality Assurance, Quality Control, Testing
Quality Assurance (QA) focuses on processes and procedures rather than conducting actual testing. It includes activities, which ensure the implementation of processes to verify the intended requirements of the developed software. It is considered a process-oriented activity as well as preventive activity.
Quality Control (QC) focuses on actual testing via execution of the software to find bugs, exploits, and vulnerabilities. QC verifies developed software with the documented requirements. It is a product-oriented activity and serves corrective purposes. It can be considered as a subset of QA.
Testing focuses on actual testing and consists of processes that try to identify all of the bugs, errors, defects, vulnerabilities, security loopholes, and etc. It is a preventative process, which is also product-oriented and is considered a subset of QC.
Inspections and Audits
As strange as it may sound, all of the formal processes to determine the integrity of the team developing the software is considered as a part of software testing.
Audits are processes, which determine if the actual testing is done in accordance with the organization’s standards. This means that there will be an independent examination of the processes, which are involved during the testing phase of the software. These may include but are not limited to Legal Compliance Audit (LCA), Internal Audit (IA), and System Audit (SA).
Inspections are done by independent teams of professionals, who conduct informal technical reviews of the software to identify any errors. This means that the whole software requirements, designs, and codes are tested in detail by a separate group of professionals other than the main team working on the actual development. The inspections are carried out to detect faults, violations, and other problems.
Debugging
Debugging is a process, which involves identifying, isolating, and fixing bugs. This is done by the developers who are writing the actual code during the writing when encountering some errors in the code.
Types of Testings
In general there are two main types of testing: manual and automated testings.
Manual Testing includes, as apparent in the name, testing the software manually. This means without any scripts which automate the process. During this the tester basically assumes the position of the end-user and goes through the whole software doing everything, which the program does not expect the user to do. There are different stages for these types of testings including and not limited to integration testing, system testing, unit testing, and user acceptance testing.
Automated testing is a process, where the testing is done using some automated system like scripts or other testing software. These scripts basically autonomously do what manual testers are doing, however, the difference is that it is much faster, improves accuracy, and saves time and money. However, manual testing is still better as “outside of the box” thinking is not something machines are capable of at the point of writing this article. Hence, they usually go through some of the most common errors and bugs but cannot exploit something they are not coded to do. Due to this, there are specific times when automation is used:
- Large projects
- Projects that require repeated testing of the same areas of code
- Projects where requirements stay the same
- Projects which require testing during higher workload as autonomous software can overload the system quickly
- Projects where the software is already stable enough for autonomous testing
Software Testing Methods
There are 3 main methods of software testing black-box testing, white-box testing, and grey-box testing.
Black-box testing is the technique of testing where the user has no prior knowledge of what the inner code of the software is. This means that the source code of the software is unavailable to the tester, hence, creating the most real-life alike situation where a malicious user is trying to find exploits in the software by interacting with the interface and testing the input and output of the system without knowing what processes the input goes through to get that specific output. This advantages of such testing shine when there is a bigger code samples, tester is not required to access the code, create the most real-life alike scenario where the user has no idea of inner workings of the software. However, the disadvantages are also present as there is limited coverage since there’s only so much you can do to test a software this way, the testing is also limited due to the fact that the tester has no idea what source code he/she is working with, the tester cannot target a specific part of the code as they have no knowledge of where and what it is, and most importantly test cases are hard to design and account.
White-box testing is an in-depth testing of the internal logic of the code. This means that the tester has access to the source code of the software and meticulously goes through every bit of it to find bugs, vulnerabilities, or breaches of expectations. The advantages of such testing are that once the tester has to look inside of the source code it becomes easier to target specific chunks of code and thus find the bugs in the application more effectively. This process also aids in optimizing the code as well as testing for extra lines of codes, which bring in other bugs. Disadvantages are present due to the fact that the tester should be very well trained. Due to the fact that this is manual testing it’s almost impossible for larger projects to test every small bit of code as it will take an exponentially huge amount of time and resources. Due to this, some parts of the code will go untested. White-box testing also requires specialized tools like debugging tools and code analyzers.
Grey-box testing is basically a mix of both white and black-box methods. The tester here has limited amount of knowledge of inner works in the code. This means that the software tester has access to design documents and the database. Due to this the tester has the ability to prepare better test data and test scenarios for the test plan.
Conclusion
Software testing is a big part of development. Without testing, one cannot roll out the final product as it will most certainly have loads of flaws, which in the end will backfire on the company providing these services. Undertested products may result in abuse of the system by the malicious users, unavailability of the service, breach of the integrity of the product, and failure of keeping the confidentiality of the userbase.
Investing in the security development of the software beforehand will save huge amounts of resources in future via foolproofing the essential parts of the software so that even if the data exploits appear and bugs cause problems they will be relatively minor and easier to control and fix in a much shorter amount of time.