{"id":12064,"date":"2025-06-13T11:11:03","date_gmt":"2025-06-13T09:11:03","guid":{"rendered":"https:\/\/www.basyskom.de\/?p=12064"},"modified":"2025-06-23T09:38:48","modified_gmt":"2025-06-23T07:38:48","slug":"improved-black-box-testing-with-cucumber-cpp","status":"publish","type":"post","link":"https:\/\/www.basyskom.de\/en\/improved-black-box-testing-with-cucumber-cpp\/","title":{"rendered":"Improved Black Box Testing with Cucumber-CPP"},"content":{"rendered":"\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Introduction<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Qt OPC UA <\/a>is a library we have started in 2015 and maintained ever since. It is covered by multiple extensive test suites written in C++ using the Qt Test<\/a>\u00a0framework. The existing test suites bring up a custom built server application based on the open62541 library and then test the API of QOpcUaClient and QOpcUaNode against that server.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Conceptually, these tests are black box tests since they are written against the public headers of Qt OPC UA. The test suites have served us well over the years, but reading the test source isn’t pleasant. Often there are a lots of implementation details which make it hard to figure out the intent of the test. After our successful evaluation<\/a> of Cucumber-JS for black box tests of an OPC UA server, we decided to have a look at\u00a0Cucumber-CPP<\/a> and what using it to build tests for the Qt OPC UA C++ API would feel like.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

We provide the full source code for the case-study project on Github<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Cucumber-CPP<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Cucumber-CPP is currently the only solution to run tests written in Gherkin using step definitions written in C++. Our first impression was that it is much less comfortable compared to other language specific implementations like Cucumber-JS because it doesn’t provide a way to parse the .feature files and execute the test scenarios itself. Instead, the intended way of using it is to build an executable which opens a TCP server with support for the JSON based cucumber wire protocol on localhost.<\/p>

\u00a0<\/p>

The official Ruby based Cucumber implementation with the\u00a0cucumber-ruby-wire plugin<\/a>\u00a0must be executed manually after starting that executable. It parses the .feature files, communicates with the server using the cucumber wire protocol, requests the execution of the step definitions and gathers the results.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Test Setup<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

We decided to implement just a few basic scenarios to test the water and not to attempt to recreate the full test coverage of the original test suites. The\u00a0examples<\/a>\u00a0from the repository served us as a good first reference on how to structure the code and which of the libraries built by the Cucumber-CPP project to link against.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Features And Scenarios<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The main focus of the test cases was to cover the basic functionalities (read\/write\/browsing\/monitored items\/method calls) superficially and to find out if common test steps could be extracted.<\/p>

\u00a0<\/p>

We ended up with six scenarios that share at least some common steps.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
Feature: Qt OPC UA Basics\n  Test basic functionality of QOpcUaClient and QOpcUaNode\n \n  Scenario: Write and read back (string)\n    Given a connected client\n    When I write "foo" to "ns=2;s=Demo.Static.Scalar.String"\n    And I read the value of "ns=2;s=Demo.Static.Scalar.String"\n    Then the value should be "foo"\n \n  Scenario: Write and read back (int64)\n    Given a connected client\n    When I write 42 to "ns=2;s=Demo.Static.Scalar.Int64"\n    And I read the value of "ns=2;s=Demo.Static.Scalar.Int64"\n    Then the value should be 42\n \n  Scenario: Browse children\n    Given a connected client\n    When I browse the children of "ns=3;s=TestFolder"\n    Then I should see that the browse result contains 86 children\n    And I should see that the browse result contains a child named "2:DateTimeArrayTest"\n \n  Scenario: Value monitoring\n    Given a connected client\n    When I write "foo" to "ns=2;s=Demo.Static.Scalar.String"\n    And I enable value monitoring for "ns=2;s=Demo.Static.Scalar.String"\n    And I wait 200 ms\n    And I write "bar" to "ns=2;s=Demo.Static.Scalar.String"\n    And I wait 200 ms\n    And I write "baz" to "ns=2;s=Demo.Static.Scalar.String"\n    And I wait 200 ms\n    Then I should have received 3 value attribute updates\n    And value update #0 should be "foo"\n    And value update #1 should be "bar"\n    And value update #2 should be "baz"\n \n  Scenario: Event monitoring\n    Given a connected client\n    When I enable event monitoring for "i=2253"\n    And I trigger an event with severity 23\n    And I trigger an event with severity 42\n    And I trigger an event with severity 5\n    Then I should have received an event with severity 23\n    And I should have received an event with severity 42\n    And I should have received an event with severity 5\n \n  Scenario Outline: Method calls\n    Given a connected client\n    When I call the multiply method with input arguments <InputA> and <InputB>\n    Then the method call should have returned Good\n    And output argument #0 should be <Product>\n \n    Examples:\n      | InputA | InputB | Product |\n      | 21     | 2      | 42      |\n      | 8      | 8      | 64      | <\/code><\/pre>