Full cycle testing includes all aspects of functional, non-functional, unit, integration, performance and regression testing both manual and using open source automation tools and technologies such as Selenium, SOAPUI, Gatling, Supertest, Nightwatch, Cucumber, and custom programming as needed. Different testing strategies need to be applied for monolithic and microservice based architecture.
Testing for Layered or Monolithic architecture: V-model verification and validation process is involved that covers different testing mechanism at different stages as follows:
- Unit test plan to perform unit testing during detailed low level design phase
- Integration test plan to perform integration testing during high level design phase
- System test plan to perform system testing during functional specification phase
- Acceptance test plan to perform acceptance testing during requirement analysis phase
Testing for Microservice architecture:Pyramid model is more effective when testing microservices due to their decentralized nature that provides loose coupling and high cohesion. This model involves testing at different stages as follows:
- Unit testing is performed on different functions of a single service
- Contract testing is performed on each microservice independently treating it as a black box
- Integration testing is performed on multiple services that have to interact with each other to fulfil a functionality
- End to End service testing is performed on multiple services that includes all external interactions such as database and third party APIs
- UI/Functional testing is done on the whole system as an end user
Products and applications are being built using polyglot persistence that involves multiple SQL & NoSQL databases to solve a use case. Testing on such diversed persistence storage systems involve good knowledge of SQL to perform complex queries on RDBMS and database proprietary query languages for NoSQL. The following tools and query languages across different databases need to be mastered to perform high quality testing:
MySQL/Oracle/Postgres: Testing using complex SQL queries, validating stored procedures, triggeres, functions, and views, checking constraints and referential integrity, and verify database schema with the reference model for data types, data length, character set, etc..
NoSQL: MongoDB query language to verify JSON documents and collections, CQL for Cassandra, Cypher QL for Neo4j, OrientDB SQL to verify nodes and relationships in OrientDB, Lucene queries to verify documents in ElasticSearch, and others
Tools:Tools such as Compass, Metabase, Talend, Zeppelin, Jupyter, and programming languages like Python and Java are extremely useful while dealing with hybrid database systems
Big Data Testing is the most complex testing process due to huge volume of data flowing thru the system that are processed by multiple tools and technologies from ingestion to analytics. It’s very important to understand the source of the data, multiple transformation and filtering process it goes thru, identifying data lineage graph, different calculations and aggregations performed on the data, and have data sense towards outliers.
Tools and technologies such as Talend, Zeppelin, Jupyter, and decent query abilities on Hive and SparkSQL are necessary to perform testing on Big Data that should cover entire data pipeline as below:
- Big Data Infrastructure Testing
- Big Data Governance Testing
- Big Data Ingestion Testing
- Big Data Profile Testing
- Big Data Extract, Load & Transform (ELT) Testing
Continuous development, integration, and deployment with containers on cloud using Agile iterative principles is the proven combination for faster time to market. Testing different moving parts at a fast rate in less than 2-week sprint cycle needs different testing strategies on top of regular full cycle testing such as below:
- Test the infrastructure & network topology and validate against server operational documents
- Automate scripts using AWS CLI & SDK to check various attributes and operations such as network in/out, S3 Object TTL, EBS snapshots, AMI frequency, and auto scaling
- Test user authentication & authorization by accessing AWS services based on IAM users
- Download and test Docker container images on multiple OS across multiple deployment environments
- Run Docker console and verify multiple container links and interaction are working as expected
- Run builds using Jenkins and validate CI & CD are working as expected
- Test blue-green deployments for continuous deployment with near zero downtime
- Test feature flags to make sure same applications behave different for different user groups
- Frontend test: Selenium web driver, PhantomJS, Nightwatch.js, Handlebars, Angular performance inspector, Audit, Network & timeline test, Webpage Test, Google PSI, Sauce Labs, Jasmine, Protractor, Karma, Navigation & routing test, Controller test, Directives test, Events dispatch test, Filters test
- Rest API test: SOAPUI, Selenium, Gatling, Apache Benchmark test, JSON payload test, Supertest, Mocha, Should, Chai
- Database test: Metabase, Zeppelin, Jupyter, SQL query, Python & shell scripts
- TMS: Zephyr, TestLink