Processes review comments.

1 files changed, 53 insertions, 9 deletions

diff --git a/content/implementation.tex b/content/implementation.tex
index d0ec21a..abd05aa 100644
--- a/content/implementation.tex
+++ b/content/implementation.tex
@@ -1,21 +1,65 @@
 \section{Implementation}
 \label{sec:implementation}
 
-A large part of this research is actually making an implementation of the protocol in such a way that it;
+A large portion of research effort was consumed by the implementation of \cmix and benchmark framework. Including setting up the benchmarking infrastructure in such a way that it;
 
 \begin{itemize}
 	\item Supports multiple but subtly different cryptographic back-ends.
-	\item Is debuggable and reusable.
-	\item Allows back-ends to be comparably benchmarked.
+	\item Is debug-able and reusable.
+	\item Allows different back-ends to be comparably benchmarked.
 \end{itemize}
 
-The following section will talk about some implementation specific things, talking about how this framework achieves these goals or why something might need some attention in future research. For more information on where to find the implementation see appendix \ref{app:impl}
+The following section will talk about some implementation specific things, talking about how this framework achieves these goals and where things could be improved. Information on where to find the implementation can be found in appendix \ref{app:impl}
+
+\subsection{Overview of the software}
+
+The software consists of 9 modules, either small libraries or separate binaries. The modules are;
+
+\begin{description}
+	\item[libcmix-crypto] Library containing the API definition of both the generic API and the specific API's for both multiplicative group and elliptic curve variants of the API. Supplying the real cryptographic operations to implement \cmix in. This library is responsible for filling the generic API with the corresponding specific API. The specific API can be chosen at project configuration time.
+	
+	\item[libcmix] This library contains the implmentation of \cmix using $libcmix{\text -}crypto$. This library only supplies \cmix related data structures and operations. The library operates by passing each call to a library function a so called $CMixContext$ structure, which in turn contains a generic API. 
+	
+	\item[libcmix-protobuf] This library contains just the \cmix definitions for the $google protobuf$ protocol. This\cmix implementation uses $protobuf$ to transmit messages between nodes and clients. This makes sure all communication between actors are properly serialized and de-serialized and allows it to inter-operate with different clients and/or nodes.
+	
+	\item[libcmix-network] Small networking library that supplies TCP servers and clients using the $Boost::asio$ library. The library supports both plain TCP and SSL connections.
+	
+	\item[libcmix-common] Small library that contains code that is relevant to both the $client$ and $node$ binaries. It contains abstractions over the $libcmix{\text -}network$ library. This was done because there is a sense of directionality in \cmix node communication, the network forms a ring where the last nodes has a connection to the first node. Therefore the library supplies Senders, Receivers and SenderReceivers. This prevents you from sending messages to the wrong node as you only receive from your previous node and only send to the next node.
+	
+	\item[liblog] Small logging library, this is just an abstraction over $Boost::Log$ and only uses the $trivial\_log$ variant.
+	
+	\item[node] A binary that implements the \cmix node behavior. Nodes can be configured at startup to fulfill certain roles. For full explanation of the configuration please refer to the \emph{-{}-help} output 
+	
+	\item[client] A binary that implements a \cmix client. Clients can also be configured as startup. Please refer to the \emph{-{}-help} output for more information.
+	
+	\item[statsd] A binary that accepts Performance statistics of the nodes via TCP. Nodes can be configured  to either connect to a stats daemon or perform operation without it.
+\end{description}
+
+\subsubsection{libcmix-crypto}
+
+One goal of $libcmix{\text -}crypto$ was to be as reusable and accessible as possible. This is why $libcmix{\text -}crypto$ was written in $C$. Another goal was to make it easy to add new cryptographic backends and an easy way to swap between them. This implementation chose to only build and use one implementation based on the configuration of the project. This was done to make sure that no code change was needed to switch between algorithms. One can easily configure the project, and which cryptographic backend to use with $CMake$. In $CMake$ one can choose between either using elliptic curve, or multiplicative group. And choose out of a list of implementations of either the elliptic curve or multiplicative group variants.
+
+For each type of encryption (multiplicative group or ed25519) multiple backends can be created and used. By adding a folder with a descriptive chosen $name$ and adding a file to that folder named $name\_algorithm.c$ for instance $gcrypt\_ed25519.c$. After rerunning the $CMake$ configuration step you can choose what backend you want to use. After a build of the project this implementation will be used as the \cmix cryptographic backend.
+
+This additional backend should initialize all the $extern$ declared variables defined in one of the specific APIs, for example $ed25519.h$ (\autoref{lst:ed25519.h}), with compatible function pointers. There are some convenience macros to help you set up additional backends.
+
+\subsubsection{libcmix}
+
+Like $libcmix{\text -}crypto$, $libcmix$ was also implemented in $C$. All $libcmix$ functions operate by taking a $CMixContext$ structure as first parameter.
+The $CMixContext$ contains all the randomness and other information needed for mix network operation. Return values from $libcmix{\text -}crypto$ are passed and stored in the opaque $GroupElement$ type. The specific cryptographic backend will be able to reinterpret objects of this type back to the right representation to do the actual cryptography with. This allows us to use the same functions (and thus function signatures) for all cryptography types and implementations.
+
+The downside is that the cryptographic backend library has to implement a kind of serialization. It is responsible for transforming $GroupElement$s to some kind of string representation and back. This representation can be, and in all current cases is, binary. Which also means that the library needs a function to let the caller know how large the serialized output will be. Furthermore the message representation does not have to match the serialized group element representation, in case of ed25519 this is impossible. So we get separate serialization for messages.
+
+It also moves responsibility of generating randomness to the cryptographic library. As we need to generate random group element vectors R and S and only the cryptographic backend library knows what those are. The library also generates the permutation function which means it needs to generate a uniform int securely, And it also implements the Diffie-Hellman key-exchange functions as it also needs a random group element to perform the key-exchange.
+
+All these extra functionality which needs to be implemented in the backend adds up and makes the API look a bit bloated, but most functions are easily implemented in a couple of lines of code. Unfortunately it is simply impossible to have a higher level library take care of these operations without losing possible abstraction of the underlying cryptographic libraries or data structure used by that library.
+
 
 \subsection{ElGamal in multiplicative Group and Elliptic Curve}
 
 The goal of this research is to see how differently these two ways of using ElGamal in this cryptographic scheme effect things like latency and throughput in \cmix. But also doing this in the most reusable way  possible so that the implementation is of use for further research.
 
-This means I will be trying to find cryptographic libraries that do most of the work while I focus on getting a uniform API over different back-ends and the correct implementation of said back-ends. Unfortunately ElGamal is not a very popular encryption algorithm in modern cryptographic libraries. and even if it is supported with a high level interface it is very rigid and wouldn't allow for any secret sharing as needed in \cmix. Because of this I needed access to lower level cryptographic primitives. Which I found in the libgcrypt library. The next step is to create my own \cmix library which abstracts the lower level cryptographic primitives for both ``multiplicative group'' and ``elliptic curve'' ElGamal. This makes it easy to switch between implementations and test without changing any of the application code. This library is written in ``C'' to make interfacing with it from other languages used in the application level easier. For the application level code I used ``C++'', but in theory this should be easily swapped for a different application language.
+This means we will be trying to find cryptographic libraries that do most of the work while we focus on getting a uniform API over different back-ends and the correct implementation of said back-ends. Unfortunately ElGamal is not a very popular encryption algorithm in modern cryptographic libraries. and even if it is supported with a high level interface it is very rigid and wouldn't allow for any secret sharing as needed in \cmix. Because of this we needed access to lower level cryptographic primitives. Which was found in the libgcrypt library. The next step is to create a \cmix library which abstracts the lower level cryptographic primitives for both ``multiplicative group'' and ``elliptic curve'' ElGamal. This makes it easy to switch between implementations and test without changing any of the application code. This library is written in ``C'' to make interfacing with it from other languages used in the application level easier. For the application level code I used ``C++'', but in theory this should be easily swapped for a different application language.
 
 Although using libgcrypt makes it possible to implement \cmix, it doesn't mean it was an easy task. The library still lacks some convenience functions that needed workarounds in the eventual implementation. This especially is the case for the elliptic curve back-end. Some specific problems will be discussed later.
 
@@ -39,13 +83,13 @@ An algorithm that might spring to mind first is the Elligator algorithm\cite{Ber
 
 \subsubsection{Koblitz's method}
 
-There does exist an algorithm to map arbitrary strings to curve points. It is a probabilistic algorithm called the Koblitz's method\cite{Koblitz}. It starts by choosing a number that will be used as a  ``stride''. The string you want to encode as a point has to be interpreted as a group element and multiplied by the stride. The result of this multiplication must also be a group element. This means your message space is made smaller by a factor ``stride''. Now you can check if your result has a corresponding $x$ coordinate, in the case of Ed25519. If it has one you are done, if it doesn't you add $1$ to your current trial $y$ coordinate and check again. This continues up until your trial $y$ coordinate reaches the value of  $first\_trial\_y\_coordinate + stride$. If you haven't found a suitable $y$ coordinate by now this mapping algorithm fails.
+There does exist an algorithm to map arbitrary strings to curve points. It is a probabilistic algorithm called the Koblitz's method\cite{Koblitz}. It starts by choosing a number that will be used as a  ``stride''. The string you want to encode as a point has to be interpreted as a group element and multiplied by the stride. The result of this multiplication must also be a group element. This means your message space is made smaller by a factor ``stride''. Now you can check if your result has a corresponding $x$ coordinate, in the case of Ed25519. If it has one you are done, if it doesn't you add $1$ to your current trial $y$ coordinate and check again. This continues up until your trial $y$ coordinate reaches the value of  $first\_trial\_y\_coordinate + stride$. If you haven't found an $x$ coordinate corresponding to your trial $y$ the mapping algorithm fails.
 
-You can map a point back to the original string by dividing the y coordinate with the ``stride''. This works because of integer division and because we only add a value less than the stride to the original number.
+You can map a point back to the original string by dividing the y coordinate with the ``stride''. This works because we only add a value less than the stride to the original number and integer division discards these additions.
 
 The problem with this probabilistic Koblitz's method is choosing your ``stride''. There is no way of knowing what the max distance between two consecutive suitable y coordinates could be. We do know that about half of the possible group elements would be suitable in Ed25519, but it is impossible to list all the Ed25519 curve points and check. This makes it an unsolvable problem, but we can make an educated guess, and stay on the safe side of that guess.
 
-Now to address the concern that you divide your message space by the stride. This reduction will theoretically effect your throughput, however it only does if you would optimally pack your messages in the 252 bit message space you have available in Ed25519. However if you only use the lower 248 bits, which gives you 31 byte message space. You have 5 bits to use as a stride. Which, anecdotally, seems to be enough. Of course you can never know if this stride will work for all possible messages. But for the purpose of this benchmark it seems to work well enough. Maybe it is possible to find out how stride influences the chance of not finding a suitable y coordinate. But that is outside of the scope of this research.
+Now to address the concern that you divide your message space by the stride. This reduction will theoretically effect your throughput, however it only does if you would optimally pack your messages in the 252 bit message space you have available in Ed25519. However if you only use the lower 248 bits, which gives you 31 byte message space. You have 5 bits to use as a stride. Which, anecdotally, seems to be enough. Of course there is no way of knowing if this stride will work for all possible messages. But for the purpose of this benchmark it works well enough. It might be possible to find out how stride influences the chance of not finding a suitable y coordinate. But that is outside of the scope of this research.
 
 A small but nice benefit of using $2^5 = 32$ as stride; multiplication and division are bit shifts as $32$ is a power of $2$. For debugging purposes you might want to consider a stride of $16$. This makes any hexadecimal representation of a number just shift up one character. However in actual runs of the algorithm with a stride of $16$ was insufficient. Some of the runs would fail because there was no suitable $y$ coordinate within $16$ value range. This has not happened for a stride of $32$ yet. To reiterate this is no guarantee that it will never happen.
 
@@ -55,4 +99,4 @@ Debugging the \cmix operations is hard. Intermediate results produced by the nod
 
 Some tools that have been a great help in creating the implementation in general are AddressSanitizer\cite{ASan} and the companion leak check tool LeakSanitizer\cite{LSan}. These tools check all the executed code paths for any object or array access outside of the allowed bounds. This is no guarantee the application is completely memory safe. It does ensure that any out of bounds access is detected, even if the access was still technically in valid memory just not part of the object or array.
 
-The leak tool also helped to keep an eye on the tokens passed around by the \cmix library. Because the implementation language of the library was $C$ and there are 2 different implementations for the 2 different ElGamal back-ends. There is a lot of token passing. These tokens need to be destructed but have no clear overarching owner. This makes it harder to keep track of object lifetime. So LeakSanitizer helped out tracking these lost tokens. Which allowed me to eventually eliminate memory leaks. Which in turn allowed me to run the benchmark for a high number of clients on a system with relatively low specs.
+The leak tool also helped to keep an eye on the tokens passed around by the \cmix library. Because the implementation language of the library was $C$ and there are 2 different implementations for the 2 different ElGamal back-ends. There is lots of token passing. These tokens need to be destructed but have no clear overarching owner at times. This makes it harder to keep track of object lifetime. So LeakSanitizer helped out tracking these lost tokens. Which allowed me to eventually eliminate memory leaks. Which in turn allowed me to run the benchmark for a high number of clients on a system with relatively low specs.