Integration testing with NodeJS
Posted by Marina Pavlik on June 29, 2023Author: Jovan Kočić, Senior Software Engineer @ Ryd
Integration testing is a software testing technique that aims to test the interaction between different software components or modules when integrated. The purpose of integration testing is to ensure that the individual components of the software system work together correctly and produce the expected results.
Integration testing is typically performed after unit testing and before system testing. The focus of integration testing is on testing the interfaces between the different components and verifying that they function correctly as a whole. Integration testing can be performed using different strategies, such as top-down integration, bottom-up integration, and incremental integration.
In top-down integration, testing starts with the highest-level modules, and lower-level modules are gradually integrated and tested. In bottom-up integration, testing starts with the lowest-level modules, and higher-level modules are gradually integrated and tested. Incremental integration involves testing the system incrementally by adding and testing new modules one at a time.
Integration testing can be done manually or using automated testing tools. Automated testing tools can help streamline the integration testing process and reduce the time and effort required for testing. However, manual testing may still be necessary for certain situations to ensure that the software is functioning as intended.
Overall, integration testing is an essential part of the software testing process and helps to ensure the overall quality and reliability of the software system.
Implementation example
For example, we can take an application with two endpoints and also consider that complete flow requires other external services.
Let’s say that we have our controller where the first method communicates with the database and the second endpoint calls external service:
// users.controller.ts
@Controller('users')
export class UsersController {
constructor(
private userService: UsersService,
private userAuthService: UserAuthService,
) {}
@Get(':id')
sss
async getById(@Param('id', new ParseUUIDPipe()) id: string) {
return this.userService.findById(id);
}
@Get('/:id/roles')
async getRolesById(@Param('id', new ParseUUIDPipe()) id: string) {
return this.userAuthService.getRolesByUserId(id);
}
}
In the service specification:
// users.service.ts
@Injectable()
export class UsersService {
constructor(private readonly userRepository: UsersRepository) {}
async findById(id: string) {
const foundUser = await this.userRepository.getById(id);
if (!foundUser) {
throw new NotFoundException('User is not found');
}
return foundUser;
}
}
External service call:
// user-auth.service.ts
@Injectable()
export class UserAuthService {
constructor(
private http: HttpService,
private configService: CustomConfigService,
) {}
// Request to the external auth service
async getRolesByUserId(userId: string) {
try {
const data = await lastValueFrom(
this.http
.get(`${this.configService.USER_AUTH_SERVICE_URL}/roles/${userId}`)
.pipe(map((res) => res.data)),
);
return data;
} catch (error) {
if (error?.response?.status) {
throw new HttpException(error?.response?.data, error.response.status);
}
throw new InternalServerErrorException();
}
}
}
Mocking external service dependency
Mocking external dependencies involves replacing the real implementation of an external service with a mock object that simulates the expected behavior of the service. This allows the integration test to continue running without relying on the actual external service.
Mocking external dependencies can be done using various tools and libraries, such as SinonJS, Nock, or Jest. In this example, we created a simple mock by using only a small Express app, since Express comes with NestJS out of the box.
First, we setup a simple server wrapper:
export const createDummyServer = async (
registerEndpoints: (Express) => void,
): Promise<DummyServer> => {
const app = express();
app.use(express.json());
registerEndpoints(app);
return new Promise((resolve) => {
const server = app.listen(0, () => {
resolve({
url: `http://localhost:${(server.address() as AddressInfo).port}`,
close: () => server.close(),
}); });
}); };
And finally, setup a mock for the user authorization service:
export const createDummyAuthServiceServer = async (): Promise&lt;DummyServer&gt; =&gt; {
return createDummyServer((app) =&gt; {
app.get('/roles/:userId', (req, res) =&gt; {
if (req.params.userId !== 'b618445a-0089-43d5-b9ca-e6f2fc29a11d') {
return res.status(404).send('User not found');
}
res.json(userRoles);
});
}); };
As you can see, we are listening for a GET request on the specified route, performing a simple check of the user id, and finally, returning the mock value from a simple JSON file.
Setup integration tests
Tests for endpoints can be split into two parts. The first is related to the external dependencies setup.
The example below creates a mocked service and spins up the database using test containers.
Testcontainers is an open-source Java-based library that allows you to run software containers in your tests. The library provides lightweight, throwaway instances of common databases, web servers, and other services that you can use for integration testing.
The idea behind Testcontainers is to make it easier to test code that interacts with external systems, without having to worry about the complexities of setting up and tearing down those systems. By using containers, Testcontainers can spin up isolated instances of databases like PostgreSQL, MySQL, and Oracle, message brokers like Kafka and RabbitMQ, and web servers like Tomcat and Jetty.
The environment variables are set for before mentioned dependencies, and the leading service starts running.
The database is cleaned before every test run. External dependencies (mocked service and database) are closed after tests finish.
describe('UsersController (integration)', () => {
let app: INestApplication;
let dummyAuthServiceServerClose: () => void;
let postgresContainer: StartedTestContainer;
let usersRepository: Repository<UsersEntity>;
const databaseConfig = {
databaseName: 'nestjs-starter-db',
databaseUsername: 'user',
databasePassword: 'some-r4ndom-pasS',
databasePort: 5432,
};
beforeAll(async () => {
const dummyAuthServiceServer = await createDummyAuthServiceServer();
dummyAuthServiceServerClose = dummyAuthServiceServer.close;
postgresContainer = await new GenericContainer('postgres:15-alpine')
.withEnvironment({
POSTGRES_USER: databaseConfig.databaseUsername,POSTGRES_PASSWORD: databaseConfig.databasePassword,
POSTGRES_DB: databaseConfig.databaseName,
})
.withExposedPorts(databaseConfig.databasePort)
.start();
const moduleFixture: TestingModule = await Test.createTestingModule({
imports: [AppModule],
})
.overrideProvider(ConfigService)
.useValue({
get: (key: string): string => {
const map: Record<string, string | undefined> = process.env;
map.USER_AUTH_SERVICE_URL = dummyAuthServiceServer.url;
map.DATABASE_HOSTNAME = postgresContainer.getHost();
map.DATABASE_PORT = `${postgresContainer.getMappedPort(
databaseConfig.databasePort,
)}`;
map.DATABASE_NAME = databaseConfig.databaseName;
map.DATABASE_USERNAME = databaseConfig.databaseUsername;
map.DATABASE_PASSWORD = databaseConfig.databasePassword;
return map[key] || '';
},
})
.compile();
app = moduleFixture.createNestApplication();
usersRepository = app.get(getRepositoryToken(UsersEntity));
await app.init();
});
beforeEach(async () => {
await usersRepository.delete({});
});
afterAll(async () => {
await app.close();
dummyAuthServiceServerClose();
await postgresContainer.stop();
});
After the test setup, we can define our tests. In the first test suite, we test retrieving data from the database, but in the second and third test suites, we assert external service calls. We are trying to cover all possible scenarios of calling the external service, in our case, it can return either a successful response or HTTP 404 error.
describe('[GET] /users/:id', () => {
it('should return found user', async () => {
const userId = 'b618445a-0089-43d5-b9ca-e6f2fc29a11d';
const userDetails = {
id: userId,
firstName: 'tester',
};
const newUser = usersRepository.create(userDetails);
await usersRepository.save(newUser);
return request(app.getHttpServer())
.get(`/users/${userId}`)
.expect(HttpStatus.OK)
.then((response) => {
expect(response.body).toEqual(userDetails);
});
});
it('should return user roles from user auth service', async () => {
const userId = 'b618445a-0089-43d5-b9ca-e6f2fc29a11d';
return request(app.getHttpServer())
.get(`/users/${userId}/roles`)
.expect(HttpStatus.OK)
.then((response) => {
expect(response.body).toEqual(userRolesResponse);
});
});
it('should return 404 error when user roles were not found', async () => {
const userId = 'b618445a-0089-43d5-b9ca-e6f2fc29a11c'; // wrong user id
return request(app.getHttpServer())
.get(`/users/${userId}/roles`)
.expect(HttpStatus.NOT_FOUND);
}); });
The complete example you can find here.
Should You Avoid Integration Tests?
Despite the disadvantages just outlined, for many users, integration tests can still be manageable with a small number of microservices, and in these situations, they still
make a lot of sense. But what happens with 3, 4, 10, or 20 services? Very quickly these test suites become hugely bloated, and in the worst case, they can result in a Cartesian-like explosion in the scenarios under test.
In fact, even with a small number of microservices, these tests become difficult when you have multiple teams sharing integration tests. With a shared integration test suite, you undermine your goal of independent deployability. Your ability as a team to deploy a microservice now requires that a test suite shared by multiple teams passes.
What is one of the key problems we are trying to address when we use the integration tests outlined previously? We are trying to ensure that when we deploy a new service to production, our changes won’t break consumers. However, schemas can’t pick up semantic breakages, namely changes in behavior that cause breakages due to backward incompatibility. Integration tests absolutely can help catch these semantic breakages, but they do so at a great cost. Ideally, we’d want to have some type of test that can pick up semantic breaking changes and run in a reduced scope, improving test isolation (and therefore speed of feedback). This is where contract tests and consumer-driven contracts come in.
Conclusion
In conclusion, NodeJS integration testing is a critical aspect of software development that helps developers to ensure the quality and reliability of their applications. By identifying and resolving issues early in the development process, developers can deliver more stable and performant applications to their users, resulting in a better user experience and improved business outcomes.
Integration testing is crucial for ensuring the quality and reliability of NodeJS applications. It helps to identify issues that may arise when different components are integrated, such as incorrect data flow or communication problems between services. By identifying and resolving these issues early in the development process, developers can avoid costly and time-consuming errors in the later stages of the application’s lifecycle.
Additionally, integration testing is important for maintaining the stability and performance of the application over time. Regular testing can help to detect issues that may arise as a result of updates or changes to external dependencies, ensuring that the application continues to function as expected.
