Detecting AI patent infringement
Finding out whether machine learning patents have been infringed is not the impossible task many believe it to be, write Kimberly Bayliss and Alexandra Roy of Haseltine Lake Kempner in this co-published article
One of the things on applicants’ minds when they are filing patents is whether the invention is detectable – after all, a patent is of limited use if you can’t show it has been infringed.
There seems to be a general assumption that machine learning (ML) models can’t be reverse engineered due to their black box nature, and that there is limited value in pursuing patents for “hidden AI”.
To get to the bottom of the detectability question, we sat down with Chris Whittleston, Delivery Lead at Faculty, one of the UK’s well-known AI companies, for an informal chat about reverse engineering. We asked him what might be detectable in different scenarios; this is a summary of what we learned.
If you can get hold of a software product, eg an executable file, then it is likely that you can get a handle on whether a machine learning model is being used
Many ML inventions fall into the category of what is known in Europe as “applied-AI”. That is, they use known ML models in an innovative way.
In this scenario, if an applied-AI invention were incorporated into a product, it is highly likely that open source libraries would be used to create and deploy the ML model. Afterall, open source libraries are reliable, well tested and, well, why re-invent the wheel? Various open-source libraries exist and the code is freely available for download.
If an open source library were used in a product, then this would be detectable through decompilation, a reverse engineering technique which recreates high-level source code that is human readable from compiled executable code (which is not human readable).
Although decompilation allows an engineer to reconstruct functions and variables called by the executable code, the original names and labels for these variables and functions are replaced with random character strings. However, the replacement is performed in a consistent manner throughout the decompiled code, meaning that variables, function names and the like are each replaced with an individual random character string.
Whilst the names and labels given to the various functions in the decompiled code may be unintelligible, the structure of these functions can be cross-referenced to known functions found in open source libraries. Thus, an engineer can recognise and identify the functions being called upon in the decompiled code.
It is therefore likely to be possible to identify known neural network code structures, such as code used to create the layers of the neural network or code used for performing common, machine learning tasks (eg, back propagation) in this way.
Regarding the identification of specific variables within the decompiled code, variables can often be traced through the various functions. This tracing can provide clues as to the datatype of the inputs to a neural network.
Thus, if a patent claim described a neural network with particular inputs, it may be possible to determine from reverse engineered code that a neural network is present in the code (if a function is identified corresponding to an open source neural network) and also to determine what inputs are being provided to it.
Outputs are usually more difficult to identify from the decompiled code, not least because custom-made code can be used to deal with this data. However, assuming you can successfully trace a given variable through the various functions applied to it, you may still be provided with sufficient information to deduce the output data type.
In conclusion, using decompilation it is likely to be possible to determine whether a feature of “using a machine learning model to predict x from y” is being infringed.
What about core-AI inventions, where the inventor has invented a new model architecture, or a new method of training a model?
Another type of ML nvention is what is referred to in Europe as a core-AI invention. These are ones where the inventor has made a fundamental improvement to the field of ML itself. For example, a new and improved model architecture or a new way of training a neural network.
These inventions are unlikely to be implemented with open source code alone because, by their very nature, they represent new ML models or processes for training ML models (eg, the types of processes that eventually end up in open source libraries).
However, it is still likely that open source libraries will be used to build up the new model – albeit with additional, bespoke code to implement the new feature. Taking an example of a new neural network having a new and improved layer, the above discussed cross-referencing technique can be applied in order to identify the various functions used to build up the new model.
In this scenario, code structures that do not match up completely to the known, open source code structures could indicate new segments of code which may have been added to implement a core-AI invention. If the functionality of the new segments of code can be inferred, then infringement of a core-AI invention may be demonstrated.
New methods of training can be identified in a similar manner – especially the functions responsible for the pre-processing of training data. That is, it is unlikely that a commercial product would be built using custom functions for standard data processing techniques (eg, standard libraries would most likely to be used to implement processes such as rotation of an image, convolution or the like) and thus different combinations of such techniques that may be used to build up a new method of training may readily be determined from decompiled code.
The presence of on-going training per se may also be inferred if a model shows drift. For example, a ML model undergoing no training (ie, a static model), when iteratively provided with the same input, would be expected to produce the same output for every iteration. On the other hand, if the iterative outputs were to drift away from the initial output, this would indicate that the model is being trained (ie, it is a non-static model).
Is it viable?
In principle, it seems that, given a compiled executable file, many commercial implementations of ML algorithms can be reverse engineered to a level that could be used to infer patent infringement. However, despite this, the time and expense required for such a process are likely to render it unviable in many circumstances.
It will also be apparent that the solutions above apply in circumstances where the executable file is available to decompile. However, many ML models will be made available as services via the cloud, their executable code effectively kept behind closed doors.
In these circumstances, the only available mechanism for gaining any information is by querying the service. In some circumstances, a targeted query pattern may be able to enable certain details about the underlying model to be obtained. For example, training data may be built up (in the manner of an extraction attack) and, in some circumstances, the type of model and architectural detail may be determined through targeted querying around a decision boundary.
Patenting AI is worthwhile
Given the complexity of the ML models currently deployed, it is easy enough to assume they simply cannot be reverse engineered. However, even from our brief discussion, it became apparent that broad, sweeping statements like this are just not true; given enough time, AI models can be reverse engineered (as is the case with many other programs).
Whether this is commercially viable on a frequent basis is another question. However,even a partial decompilation may be sufficient to justify bringing legal proceedings, especially given that the disclosure phase of litigation could be used to obtain fuller information.
Finally,, even if the time and cost factors of reverse engineering are off-putting, this shouldn’t dissuade an applicant from patenting AI, as future developments in AI regulation and standardisation will affect the ability of commercial ML models to remain hidden. For example, the European Union is actively working on proposals to regulate AI. These are likely to require the inner workings of AI products used in certain industries to be made transparent in order to ensure they meet European regulatory stipulations.
Therefore, applicants should carefully consider whether they wish to forgo patent protection for their AI inventions based on a generalised assumption that all AI infringement is undetectable.
Kimberly Bayliss is a senior associate in Haseltine Lake Kempner’s AI team and Alexandra Roy is a trainee patent attorney in the firm’s tech team
Previous articles by Haseltine Lake Kempner authors in this series can be accessed here:
How to secure AI patents in Europe
Drafting AI patent applications for success at the EPO – eligibility and claim formulation
Drafting AI patent applications for success at the EPO – drafting the full specification
Technology trends – why patent your hidden AI?
Google and Samsung top the list of applicants for AI-related patents at the EPO
The EPO and UKIPO approaches to AI and patentable subject matter
How revised EPO guidelines affect treatment of AI inventions