Implementing source code coverage for Swift step by step

introduce

Recently, I have been researching solutions for Swift-based code coverage detection solutions for my company's projects. I have tried hard and found the best practices.

In this short article, I will introduce you to:

• How to generate *.profraw files and measure code coverage via command line
• How to call C/C++ methods in Swift App projects
• How to measure code coverage for a complete Swift App project in Xcode

Practice using the command line

Before we measure the code coverage of a complete App project, we need to create a simple Swift source code file and generate a ​*.profraw​ file using the command line so that we can learn the basic workflow of generating a coverage profile.

Create a Swift file and include the following code:

 test ( )
 print ( "hello" )
 func test ( ) {
 print ( "test" )
 }
 func add ( _ x : Double , _ y : Double ) - > Double {
 return x + y
 }
 test ( )

Run the following command in the terminal:

 swiftc - profile - generate - profile - coverage - mapping hello .swift

The options ​-profile-generate​​​​ and ​-profile-coverage-mapping​​​ passed to the compiler will enable the coverage feature when compiling source code. Source-based code coverage operates directly on the AST and preprocessor information.

Then run the output binary:

 ./hello ​ ​

After the run is complete, execute ​ls​​​​ in the current directory, and we will see that a new file named ​default.profraw​​​ is generated here. This file is generated by llvm. In order to measure code coverage, we must use another tool llvm-profdata to combine multiple raw profiles and index them at the same time.

 xcrun llvm - profdata merge - sparse default .profraw - o hello .profdata

Running the above command line in the terminal, we will get a new file named ​hello.profdata​ ​​, which can show the coverage report we want. We can use llvm-cov to display or generate JSON reports.

 xcrun llvm - cov show ./hello - instr - profile = hello .profdata ​ ​
 xcrun llvm - cov export ./hello - instr - profile = hello .profdata ​ ​

Now, we have learned the basic workflow of generating Swift code coverage reports. It seems that Swift source-based code coverage is not that difficult. However, the configuration of a complete Swift App project in Xcode is very different from the command line. Let's continue reading!

Measuring Code Coverage for Swift App Projects in Xcode

Creating a Swift Project

Select ​SwiftCovApp target -> Build Settings -> Swift Compiler — Custom Flags​ .

Add ​-profile-generate​​​​ and ​-profile-coverage-mapping​​​ options to Other Swift Flags:

If we try to compile now, we will get the following error message:

To fix this, we have to enable code coverage for all targets:

After enabling code coverage and running again, the project will build successfully.

We learned that when the program exits, the compiler will write the original profile to the path specified by ​LLVM_PROFILE_FILE​​​​ environment variable. So we should kill the Application process to achieve the ​*.profraw​​​ file. However, when we end the application, it reports an error in the console:

Although I set the same configuration in Build Settings, the default environment path in Xcode is empty. To solve this problem, we must create a new header file and declare some llvm C api functions for Swift to call.

Calling C/C++ methods from Swift

Swift is a powerful language based on C/C++, which can call C/C++ methods directly. However, before we call the llvm C/C++ api, we must export the methods we need as a module.

First, create a header file:

Then, copy and paste the following code into the file:

 #ifndef PROFILE_INSTRPROFILING_H_
 #define PROFILE_INSTRPROFILING_H_int __llvm_profile_runtime = 0 ; void __llvm_profile_initialize_file ( void ) ;
 const char * __llvm_profile_get_filename ( ) ;
 void __llvm_profile_set_filename ( const char * ) ;
 int __llvm_profile_write_file ( ) ;
 int __llvm_profile_register_write_file_atexit ( void ) ;
 const char * __llvm_profile_get_path_prefix ( ) ; #endif /* PROFILE_INSTRPROFILING_H_ */

Create a ​module.modulemap​​​ file and export everything as a module.

 //
 // module.modulemap
 //
 // Created by yao on 2020/10/15.
 //

 module InstrProfiling {
 header "InstrProfiling.h"
 export *
 }

Actually we can’t create ​module.modulemap​​ ​​directly, first create a ​module.c​​​​ file and rename it to ​module.modulemap​​ ​​, it also helps me create a ​SwiftCovApp-Bridging-Header​​​ file.

Build the project, then we can call llvm apis in Swift code.

 import UIKit
 import InstrProfiling
 class ViewController : UIViewController {
 override func viewDidLoad ( ) {
 super .viewDidLoad ( )
 // Do any additional setup after loading the view.
 print ( "File Path Prefix: \( String ( cString : __llvm_profile_get_path_prefix ( ) ) )" )
 print ( "File Name: \( String ( cString : __llvm_profile_get_filename ( ) ) )" )
 let name = "test.profraw"
 let fileManager = FileManager .default

 do {
 let documentDirectory = try fileManager .url ( for : .documentDirectory , in : .userDomainMask , appropriateFor : nil , create : false )
 let filePath : NSString = documentDirectory .appendingPathComponent ( name ) .path as NSString
 __llvm_profile_set_filename ( filePath .utf8String )
 print ( "File Name: \( String ( cString : __llvm_profile_get_filename ( ) ) )" )
 __llvm_profile_write_file ( )
 } catch {
 print ( error )
 }
 }
 }

Build and start the App and we will see the original configuration file path in the console.

Finally, we have the original configuration file we need! 🎉

We can copy this file and the Mach-O (binary file) in the Swift App project to the temp directory so that we can check whether the configuration file can generate the correct report.

Create a new Swift file:

 import Foundation
 struct BasicMath {
 static func add ( _ a : Double , _ b : Double ) - > Double {
 return a + b
 }
 var x : Double = 0
 var y : Double = 0
 func sqrt ( _ x : Double , _ min : Double , _ max : Double ) - > Double {
 let q = 1e-10
 let mid = ( max + min ) / 2.0

 if fabs ( mid * mid - x ) > q {
 if mid * mid < x {
 return sqrt ( x , mid , max )
 } else if mid * mid > x {
 return sqrt ( x , min , mid )
 } else {
 return mid
 }
 }
 return mid
 }
 func sqrt ( _ x : Double ) - > Double {
 sqrt ( x , 0 , x )
 }
 }

Add the call to sqrt before the call to __llvm_profile_write_file in ViewController.swift. Then, build and run.

 print ( "√2=\( BasicMath ( ) .sqrt ( 2 ) )" )
 __llvm_profile_write_file ( )

Run the following command in the command line:

 mkdir TestCoverage
 cd TestCoverage
 cp / Users / yao / Library / Developer / CoreSimulator / Devices / 4545834 C -8 D1F -4 D2C - B243 - F9E617F6C52D / data / Containers / Data / Application / 6 AEFAB1B - DA52 -4 FAF -9 B27 -3 D47A898E55C / Documents / test .profraw .
 cp / Users / yao / Library / Developer / Xcode / DerivedData / SwiftCovApp - bohvioqnvkjxnnesyhlznzvmmgcg / Build / Products / Debug - iphonesimulator / SwiftCovApp.app / SwiftCovApp . ​
 ls
 xcrun llvm - profdata merge - sparse test .profraw - o test .profdata
 xcrun llvm -cov show ./SwiftCovApp -instr -profile = test .profdata ​ ​ ​ ​ ​

We will see the final report.

refer to

• Clang 12 Documentation.
• ​​Exploration of precise testing of mixed Objective-C and Swift projects. ​​