-
Notifications
You must be signed in to change notification settings - Fork 5
Home
The main objective of 3FD is to help C++ programmers to code their applications faster writing less boiler plate code. These are the main features provided by the framework:
- An error handling infrastructure with stack tracing
- An easy to use command line parser
- A simplified model for heterogeneous programming backed by OpenCL
- A OOP+RAII model for relational data access backed by SQLite embedded engine
- A OOP+RAII model for ISAM data access backed by Microsoft ESE
- Helpers for quick setup and use of Microsoft SQL Server Service Broker queues
- Helpers & wrappers around Microsoft WWS API, for quick development of SOAP web services
- Helpers & wrappers around Microsoft RPC API, for quick development of RPC servers and clients
- Garbage collection
Previous versions of the framework were available as compiled DLL's deployed by an msi installer, and that was part of the mentality when the framework was thought to behave like a COM component. However, after working with portions of the framework on the field, I came to the conclusion that C++ programmers are better served with a distributed source code of a highly modularized framework solution.
Basically, you download the source code where the main project is the framework. For Windows platform, Visual Studio 2013 (can be Express) or later is required. A port for POSIX is available and you can take a look here. Because this project is a static library, your solution can link to it, and only what is necessary is ever linked to your final build. Projects for unit and integration tests are also present in the source.
3FD makes use of both POCO and Boost, so you need to have them in your system. Also, in Windows Visual Leak Detector is used to control memory leaks, but you can rule this out of the dependencies if you want, by just erasing the #include <vld.h> in preprocessing.h.
Applications built using the framework require at least Windows Vista (or Server 2008). The main reason for such dependency is because these Windows versions introduced several improvements in the Win32 API that help this framework to be faster.
The project is currently being developed in Visual Studio 2015 Community Update 3 and it uses C++11 language features like lambdas and variadic macros. That's why the client application has to be compiled in a C++11 compliant compiler. Currently, the only compilers I know to be capable of it are the Microsoft Visual C++ 2013 (or later, including Express and Community editions) and Clang v3.3 (or later).
In the early stages of this project, the latest distribution of MinGW for Windows was based on GCC 4.7, which is not compatible. They were not C++11 compliant enough, rather because of the Standard Library implementation which is a little outdated compared to MSVC. The same problem was verified in Linux. Because of that, Clang was chosen in the POSIX port of 3FD.
This is optional, but you might want to run the integration tests in order to test how compliant to the framework requirements your environment is. The modules that require special attention are the OpenCL, Microsoft RPC and WWS API wrappers and MSSQL broker helpers.
The integration tests for RPC and WWS API wrappers are split into server (IntegrationTests project, where the tests for all other modules also reside) and client (TestRpcClient and TestsWwsClient projects) applications. All the applications must be started, but the client applications stay on hold, waiting the servers to become available before firing their requests (or they time out, if that takes too long).
Luckily, in Windows it is pretty straightforward to have OpenCL enabled applications work. All you need is a compliant driver for you GPU. However, the driver packages of some GPU's provide incomplete support to OpenCL or even none at all. In those cases, you can still make most of the integration tests pass as long as you change the accelerator device to be your CPU. That can be done by changing the key testOclUseGpuDevice in IntegrationTests.exe.3fd.config.
Notice that even when you set everything right, some tests might not pass. For example, I have an Intel Core i5 machine whose integrated GPU cannot perform most of the buffer mapping operations. I guess these are short-comings of the OpenCL implementations of some vendors. (Normally, AMD hardware devices are the most compliant with Khronos OpenCL standard.)
The tests cover several scenarios with different security set-ups. The security set-up for NTLM and Kerberos SSP's is selected when the test applications are built, by choosing one of the macros in TestShared\rpc_test_shared.h:
- SCENARIO_SINGLE_BOX_LOCAL_SEC is the most simple scenario, requires all test applications to run in a single box using Windows local security for authentication (NTLM) of calls using both local and TCP protocol sequences
- SCENARIO_SINGLE_BOX_AD_SEC also has all test applications running in a single box, but there is availability of Microsoft Active Directory Services for authentication (Kerberos) of calls using both local and TCP protocol sequences
- SCENARIO_REMOTE_WITH_AD_SEC uses only TCP protocol sequence, hence allowing the client and test applications to run in separate boxes and authenticate using Microsoft Active Directory Services (Kerberos)
You are also required to specify you RPC server location and its SPN (registered Server Principal Name, which unless set otherwise is the FQDN of the user account running IntegrationTests.exe) in the keys testRpcServerPrincipalName, testRpcServerSingleBox and testRpcServerWithADSec of TestRpcClient.exe.3fd.config.
For the SCHANNEL test case you must set up client and server side certificates. For Windows 8.1 and later you can do it with a Powershell prompt elevated with admin privileges:
(in server machine)
New-SelfSignedCertificate -Type SSLServerAuthentication -Subject "CN=MySelfSignedCert4DevTestsClient" -Provider "Microsoft Strong Cryptographic Provider" -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.2","2.5.29.17={text}upn=john.smith@outlook.com") -KeyUsage DataEncipherment -KeyAlgorithm RSA -KeyExportPolicy ExportableEncrypted -KeyLength 2048 -CertStoreLocation "Cert:\LocalMachine\My" -NotAfter $(get-date -year 2026)
(in client machine)
New-SelfSignedCertificate -Type SSLServerAuthentication -Subject "CN=MySelfSignedCert4DevTestsServer" -Provider "Microsoft Strong Cryptographic Provider" -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.1","2.5.29.17={text}upn=john.smith@outlook.com") -KeyUsage DataEncipherment -KeyAlgorithm RSA -KeyExportPolicy ExportableEncrypted -KeyLength 2048 -CertStoreLocation "Cert:\LocalMachine\My" -NotAfter $(get-date -year 2026)
In case you are running the tests in an older Windows version, you will need to find a Win8.1+ box to generate a single certificate with the command
New-SelfSignedCertificate -Type SSLServerAuthentication -Subject "CN=MySelfSignedCert4DevTests" -Provider "Microsoft Strong Cryptographic Provider" -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.2,1.3.6.1.5.5.7.3.1","2.5.29.17={text}upn=john.smith@outlook.com") -KeyUsage DataEncipherment -KeyAlgorithm RSA -KeyExportPolicy ExportableEncrypted -KeyLength 2048 -CertStoreLocation "Cert:\LocalMachine\My" -NotAfter $(get-date -year 2026)
than you can export it to a pfx file (including the private key) and import in your test machine(s).
(Notice that in parameter Subject we use the same name specified in source files IntegrationTests\tests_rpc.cpp and TestRpcClient\tests_rpc.cpp.)
The issued certificates are not automatically taken as trustworthy. So you will have to copy them to "Trusted Root Certification Authorities\Certificates" (in management console).
Finally, having all the applications built with those changes and certificates in place, remember to allow all TCP connections to IntegrationTests.exe in your firewall.
Wrappers for WWS API set up the web server using the provided WSDL. You must specify valid addresses for the endpoints in IntegrationTests\calculator.wsdl at //wsdl:definitions/wsdl:service/. Then you use the same addresses in the keys testWwsHostUnsecEndpoint, testWwsHostSslEndpoint and testWwsHostHAuthEndpoint of TestWwsClient.exe.3fd.config.
For transport security, again you need to set up some certificates. In the Visual Studio developer prompt, run
makecert -pe -n "CN=Certification Authority for Development" -cy authority -ss root -sr LocalMachine -r
This creates a certification authority and install it under "Trusted Root Certification Authorities\Certificates" (in management console). Then, use the new CA to create a new certificate for your local machine. Notice you must use your computer name in the commands below:
(n server machine)
makecert -pe -n "CN=MyComputerName" -ss My -sr LocalMachine -in "Certification Authority for Development" -is root -ir LocalMachine -eku 1.3.6.1.5.5.7.3.1
(in client machine)
makecert -pe -n "CN=MyComputerName" -ss My -sr LocalMachine -in "Certification Authority for Development" -is root -ir LocalMachine -eku 1.3.6.1.5.5.7.3.2
(In a development environment confined within a single box, you can choose to have only one certificate for both client and server authentication, hence eliminating the "-eku" option.)
Because both client and server must rely in the same certificate CA, when you create the CA in one box, you must export the certificate to a pfx file (including the private key) and import it into the other box. This way, the certificates in both boxes will have been issued by a common trusted CA.
In the server machine, go to the Windows management console (mmc.exe) and use the snap-in for certificate management to look for the certificate you have just created. The thumbprint is the hash you will use in the command below. These commands will assign the new certificate to the endpoints localhost:8888 and localhost:8989:
netsh http add sslcert ipport=0.0.0.0:8888 certhash=8f43288ad272f3103b6fb1428485ea3014c0bcfe appid={04EBD759-F3FF-4992-8D02-C1BFDB027A58} clientcertnegotiation=enable
netsh http add sslcert ipport=0.0.0.0:8989 certhash=8f43288ad272f3103b6fb1428485ea3014c0bcfe appid={0670F436-50EA-4B8A-B62A-02EF8B39F9C9} clientcertnegotiation=enable
(The application ID can be any GUID.)
In TestWwsClient.exe.3fd.config the same hash must be set to key testWwsCliCertThumbprint. Finally, having all the applications built with those changes and certificates in place, remember to allow all TCP connections to IntegrationTests.exe in your firewall.
The helpers for Microsoft SQL Server Service Broker rely on an ODBC connection to the SQL Server instance. For that to work, you need to make sure that
- The SQL Server instance allows connections via TCP (take note of the port, which is 1433 by default)
- The firewall allows incoming connections to such TCP port (by default, Windows Firewall does not)
- You have created the database and login "tester" for tests, which can be done by running the script CreateMsSqlSvcBrokerDatabase.sql
- You have installed the Microsoft ODBC driver for SQL Server
- The connection string is right in the key testBrokerWindowsMsSqlDbConnString of IntegrationTests.exe.3fd.config, which means there you need to have the version of the driver you installed and the network address of you SQL Server instance (including the TCP port)
First you need to compile the framework. For that you just need to run the solution build script for Powershell (build.ps1). Remember POCO (libraries Foundation, XML, Util and Data/ODBC) and Boost are dependencies that must be present and built by your development environment. In order to keep your applications less dependents on third party libraries to be carried around, I made a practical choice to use both Boost and POCO as static libraries. It is a little bit of work to build all of them from source, so I automated this work as one of the initial tasks of the build scripts. If you let it, those dependencies will be built and installed in the location specified by the environment variables BOOST_HOME and POCO_ROOT that you MUST set like this:
BOOST_HOME = C:\Program Files\Boost\v1.65.1
POCO_ROOT = C:\Program Files\POCO\v1.7.9\
Pay attention! The location is of your choice, but the version number in the path MUST be the same currently in use by 3FD project, otherwise the build script WILL FAIL. Had everything went good, you only need to use the build script like this:
.\build.ps1 -arch {x86|x64|ARM|all} -xp {yes|no} -rebuild {yes|no}
It should take a while, but in the end you have in the "build" folder that has everything you need (headers, compiled libraries and symbols) to use the framework in your projects.
The procedure for the POSIX port is here.
Once the framework is compiled, you are going to set up your solution to link to the framework just like you do for any other static library. You should also set you Visual C++ projects to use those environment variables for finding POCO and Boost installation. Then there are just a couple of things to set:
-
Define the macro ENABLE_3FD_CST in your environment so as to enable the stack tracing feature in your release (recommended);
-
Provide a XML configuration file for the framework instance used by you application. This file must have the same name of the application executable (including the extension) plus ".3fd.config". A template for this file is provided in the source.
Now you are ready to go!
For a quick start, please refer to the source code where you can find examples of utilization:
SOURCE CODE >> Files >> IntegrationTests | TestRpcClient | TestWwsClient | VideoTranscoder | ImageTranscoder
This is rather the integration tests used in the framework development. Because it covers all important features of 3FD, newcomers can use it as a pretty comprehensive sample code. (The googletest framework syntax for "assert's" and "expect's" do obfuscate it a litte bit, but not that much.)
Also, every application that makes use of 3FD needs to provide a XML file that configures the framework instance in use by the application. This file is of utter importance. It controls the behavior of all the components of the framework instance, from GC tunning to error reporting settings. For an example, please refer to:
SOURCE CODE >> Files >> IntegrationTests >> application.config