Use functional Swift to convert images to characters

Today, I was sorting out the articles to be read in Pocket and saw this article "Creating ASCII art in functional Swift", which explains how to use Swift to convert images into ASCII characters. The specific principle is explained in detail in the article, so I will not repeat it here, but the "in functional Swift" in the title made me very interested. I wanted to know where the functionality is reflected, so I downloaded the swift-ascii-art source code to find out.

Pixel

The image is composed of pixels, which are implemented by the Pixel struct in the code. Each pixel is allocated 4 bytes, which (2^8 = 256) are used to store the RBGA value.

createPixelMatrix

You can create a width * height pixel matrix using the static method createPixelMatrix:

 static func createPixelMatrix(width: Int, _ height: Int) -> [[Pixel]] {
 return map( 0 .. map( 0 .. let offset = (width * row + col) * Pixel.bytesPerPixel
 return Pixel(offset)
 }
 }
 }

Unlike the traditional method of using a for loop to create a multidimensional array, this is achieved through the map function. In Swift 2.0, the map function has been removed and can only be called as a method.

intensityFromPixelPointer

The intensityFromPixelPointer method calculates and returns the brightness value of a pixel. The code is as follows:

 func intensityFromPixelPointer(pointer: PixelPointer) -> Double {
 let
 red = pointer[offset + 0 ],
 green = pointer[offset + 1 ],
 blue = pointer[offset + 2 ]
 return Pixel.calculateIntensity(red, green, blue)
 }
 private   static func calculateIntensity(r: UInt8, _ g: UInt8, _ b: UInt8) -> Double {
 let
 redWeight = 0.229 ,
 greenWeight = 0.587 ,
 blueWeight = 0.114 ,
 weightedMax = 255.0 * redWeight +
 255.0 * greenWeight +
 255.0 * blueWeight,
 weightedSum = Double(r) * redWeight +
 Double(g) * greenWeight +
 Double(b) * blueWeight
 return weightedSum / weightedMax
 }

The calculateIntensity method obtains the intensity of a pixel based on Y'UV encoding:

 Y' = 0.299 R' + 0.587 G' + 0.114 B'

YUV is a color encoding method, where Y represents brightness, UV represents color difference, and U and V are the two components that make up color. Its advantage is that it can use the characteristics of the human eye to reduce the storage capacity required for digital color images. The Y we get from this formula is the value of brightness.

Offset

There is actually only one value stored in Pixel: offset. The matrix created by Pixel.createPixelMatrix looks like this:

 [[ 0 , 4 , 8 , ...], ...]

It does not store the relevant data of each pixel as you might imagine, but is more like a conversion tool that calculates the grayscale value of PixelPointer.

AsciiArtist

AsciiArtist encapsulates some methods for generating character paintings.

createAsciiArt

The createAsciiArt method is used to create character art:

 func createAsciiArt() -> String {
 let
 // Load image data and get pointer object  
 dataProvider = CGImageGetDataProvider(image.CGImage),
 pixelData = CGDataProviderCopyData(dataProvider),
 pixelPointer = CFDataGetBytePtr(pixelData),
 // Convert the image into a brightness matrix  
 intensities = intensityMatrixFromPixelPointer(pixelPointer),
 // Convert the brightness value into the corresponding character  
 symbolMatrix = symbolMatrixFromIntensityMatrix(intensities)
 return join( "\n" , symbolMatrix)
 }

The CFDataGetBytePtr function returns the pointer to the byte array of the image. Each element in the array is a byte, that is, an integer from 0 to 255. Every 4 bytes form a pixel, corresponding to the RGBA value.

intensityMatrixFromPixelPointer

intensityMatrixFromPixelPointer This method generates the corresponding brightness value matrix through PixelPointer:

 private func intensityMatrixFromPixelPointer(pointer: PixelPointer) -> [[Double]]
 {
 let
 width = Int(image.size.width),
 height = Int(image.size.height),
 matrix = Pixel.createPixelMatrix(width, height)
 return matrix.map { pixelRow in
 pixelRow.map { pixel in
 pixel.intensityFromPixelPointer(pointer)
 }
 }
 }

First, an empty two-dimensional array is created through the Pixel.createPixelMatrix method to store values. Then two maps are nested to traverse all the elements in it and convert pixels into intensity values.

symbolMatrixFromIntensityMatrix

The symbolMatrixFromIntensityMatrix function converts an array of brightness values ​​into an array of glyphs:

 private func symbolMatrixFromIntensityMatrix(matrix: [[Double]]) -> [String]
 {
 return matrix.map { intensityRow in
 intensityRow.reduce( "" ) {
 $ 0 + self.symbolFromIntensity($ 1 )
 }
 }
 }

Map + reduce successfully implements string accumulation. Each reduce operation uses the symbolFromIntensity method to obtain the character corresponding to the brightness value. The symbolFromIntensity method is as follows:

 private func symbolFromIntensity(intensity: Double) -> String
 {
 assert ( 0.0 <= intensity && intensity <= 1.0 )
 let
 factor = palette.symbols.count - 1 ,
 value = round(intensity * Double(factor)),
 index = Int(value)
 return palette.symbols[index]
 }

Pass intensity, make sure the value is between 0 and 1, convert it to the corresponding character through AsciiPalette, and output sumbol.

AsciiPalette

AsciiPalette is a tool used to convert numerical values ​​into characters, just like a palette in a character painting, generating characters based on different colors.

loadSymbols

loadSymbols loads all symbols:

 private func loadSymbols() -> [String]
 {
 return symbolsSortedByIntensityForAsciiCodes( 32 ... 126 ) // from ' ' to '~'  
 }

As you can see, the character range we selected is 32 ~ 126 characters. The next step is to sort these characters by brightness through the symbolsSortedByIntensityForAsciiCodes method. For example, the & symbol definitely represents an area darker than ., so how is it compared? Please see the sorting method.

symbolsSortedByIntensityForAsciiCodes

The symbolsSortedByIntensityForAsciiCodes method implements string generation and sorting:

 private func symbolsSortedByIntensityForAsciiCodes(codes: Range) -> [String]
 {
 let
 // Generate character array through Ascii code for backup  
 symbols = codes.map { self.symbolFromAsciiCode($ 0 ) },
 // Draw the characters and convert the character array into a picture array for brightness comparison  
 symbolImages = symbols.map { UIImage.imageOfSymbol($ 0 , self.font) },
 // Convert the image array into a brightness value array. The brightness value is expressed as the number of white pixels in the image.  
 whitePixelCounts = symbolImages.map { self.countWhitePixelsInImage($ 0 ) },
 // Sort the character array by brightness value  
 sortedSymbols = sortByIntensity(symbols, whitePixelCounts)
 return sortedSymbols
 }

Among them, the sortByIntensity sorting method is as follows:

 private func sortByIntensity(symbols: [String], _ whitePixelCounts: [Int]) -> [String]
 {
 let
 // Use the dictionary to establish the relationship between the number of white pixels and the character  
 mappings = NSDictionary(objects: symbols, forKeys: whitePixelCounts),
 // Remove duplicates from the array of white pixel numbers  
 uniqueCounts = Set(whitePixelCounts),
 // Sort the array of white pixel numbers  
 sortedCounts = sorted(uniqueCounts),
 // Using the previous dictionary mapping, convert the sorted number of white pixels into corresponding characters, thereby outputting an ordered array  
 sortedSymbols = sortedCounts.map { mappings[$ 0 ] as! String }
 return sortedSymbols
 }

summary

After a brief review of the project, I can vaguely feel some functional style, which is mainly reflected in the following aspects:

The application of functions such as map reduce is just right, and can easily handle the conversion and concatenation of arrays.

Data processing is performed through input and output, such as the sortByIntensity method and the symbolFromIntensity method.

There are few states and attributes, but more direct function conversions. Function logic does not depend on external variables, but only on the parameters passed in.

The code feels simple and light. Through this simple example, we can verify what we have learned from the functional features.

It feels great!