Deep Learning: Tracking the Growth of an Emerging Technology

“‘Deep learning’ is an emerging technology that not only shows traction, but explosive patent growth as well.”

In 2015, I spotted what I thought might be an emerging technology: deep learning. Because of my engineering education, I was able to go up the “deep learning” curve. The term “deep learning” is the current name for a “deep neural network,” which was previously called a “multi-layer neural network.”

While our organic brains are filled with approximately 86 billion neurons, the “deep learning” quest was built on mathematics and Graphics Processing Units (GPUs). It seemed like a breakthrough. With enough examples of a category, could a deep learning model assess data that the model hadn’t seen previously, and then score, rank and report the “matches” to a user? In short order, I was persuaded that the answer was yes.

Early Indicators

On June 25, 2016, I read the opinion in Bascom Global v. AT&T Mobility and learned that an “inventive concept” would satisfy “step two” in Alice v. CLS Bank. Which meant, theoretically, that even software systems using the technology would pass muster for a patent. But how can anyone tell whether an emerging technology like deep learning is gaining traction?

An early indicator would be what experts in the field say publicly. On September 28, Fortune published “Why Deep Learning Is Suddenly Changing Your Life – And will soon transform corporate America,” by Roger Parloff. In that article, Andrew Ng, a former computer science professor at Stanford, compared the changes brought about by electricity during the Industrial Revolution to the type of change deep learning would herald.

Out of curiosity, I started keeping track of patents issued by the USPTO where the terms “deep learning,” “deep neural” or “multi-layer neural” were used in the claims. The USPTO publishes new patents every Tuesday. So, every Tuesday, I still run the same search, using the Advanced search framework, where ALCM is the acronym for Claims and ISD is the acronym for Issue Date. My search terms are always the same (capitalization is not necessary but helps for visualization):

ACLM/(“deep learning” or “deep neural” or “multi-layer neural”) and ISD/2021.

The results are presented in only a minute or so. The annual total is identified, and all of the issued patents are listed, but (fortunately) the USPTO output starts with the most recent 50 patents.

Gaining Traction

I started in 2017. Later, I realized I could go back in time because the numbers would be low. What I found was that the total for 2016 was 39, which was not exciting. But the total for 2017 surpassed the 2016 total and ended at 83, and more than double the 2016 total. The total of 83 in 2017 was surpassed by the total in 2018, which was 162. The jump wasn’t more than double the 2017 number, but it was close to double. I was seeing traction.

By 2018, deep learning was being understood as a way for computers to make predictions. In every realm of uncertainty, deep learning for both images and language was leading to innovations. Three economists at the University of Toronto, in a book entitled Prediction Machines, put their finger on why: deep learning was reducing the cost of prediction under uncertainty. To summarize from a review: “Prediction tools increase productivity–operating machines, handling documents, communicating with customers. Uncertainty constrains strategy. Better prediction creates opportunities for new business structures and strategies to compete.”

For example, supply chains “think” in real time; self-driving vehicles and robots respond to risks in a changing environment; and social media companies use examples of “hate speech” to take it down – if it’s identical or nearly identical to the examples flagged by “content moderators” – in milliseconds and before anyone reads the text that’s against a company policy.

Then, sure enough, the total for 2019 was 396, and the total for 2020 was 726. Then, I realized I could go back in time to find out what was leading up to the results I had surfaced. I ran the same search again, starting with ISD/2001. There was only one patent where one of the three terms were in the claims. In 2002, there were four.

The Data

The bar chart below shows the following results after that: in 2003(1); 2004(0); 2005(0); 2006(1); 2007(1); 2008(0); 2009(0); 2010(0); 2011(1); 2012(1); 2013(3); 2014(4) and 2015(4). As seen on the chart, even four patents are barely noticeable.

But then look at how “deep learning” began to show that a “land rush” for patents was going on. As of the end of Q3 on September 28, 2021, the bar chart looks like this, with the total in 2021 (so far) at 824 new “deep learning” patents. Clearly, the conclusion is that “deep learning” is an emerging technology that not only shows traction, but explosive patent growth as well. And I can report that U.S. patents are going to U.S. inventors as well as assignees from many other countries. Given the rate at which I see new patents every week, I project a Year End total of 1,099.

 

Share

Warning & Disclaimer: The pages, articles and comments on IPWatchdog.com do not constitute legal advice, nor do they create any attorney-client relationship. The articles published express the personal opinion and views of the author as of the time of publication and should not be attributed to the author’s employer, clients or the sponsors of IPWatchdog.com.

Join the Discussion

5 comments so far.

  • [Avatar for Nick Brestoff]
    Nick Brestoff
    October 3, 2021 12:41 pm

    To @Anon — In my reply Comment #3, I wrote that the total from January 1, 2001 to September 28, 2021 was 2,191. I erred. I ran a search in the USPTO that had no date and the result was 2,270. But when I added up the yearly results beginning with 2001, the total was only 2,253. I wondered why, so I went year by year from 1991 to 2000, and found the missing 17. So now the Bar Chart for “deep learning (2001 to September 28, 2021) is correct.

  • [Avatar for B]
    B
    September 30, 2021 05:55 pm

    @ Anon “‘Getting’ is only the very beginning.

    Totally agree. TC 2100 isn’t completely insane like a certain other TC.

    That said – certain judges at the CAFC will declare, sans evidence but with a bizarrely stupid legal theory, that said claims lack an inventive concept, you know, because “mental steps” and all.

    Dear Nick – good article. I love seeing this type of data. I look forward to more from you.

  • [Avatar for Nick Brestoff]
    Nick Brestoff
    September 30, 2021 04:11 pm

    FYI, I forgot to mention that the current total from January 1, 2001 to September 28, 2021 is 2,191. Nick

  • [Avatar for Nick Brestoff]
    Nick Brestoff
    September 30, 2021 04:02 pm

    Hi Anon, I’ve thought about your comment. Maybe “getting” is the first dot, and an academic may have the time and resources to gather “grant” rates, PGR data, etc. I will be pleased to cooperate and here’s what I can add — for the patents in the Bar Chart, i.e., the patents that use “deep learning” in the claims, I have them ALL on a spreadsheet in NUMERICAL order.

  • [Avatar for Anon]
    Anon
    September 30, 2021 10:06 am

    It would be ‘interesting’ to compare the “grant” rates with viability indicators such as Post Grant Review rescissions, attempted (but not successful) enforcements, and the like.

    “Getting” is only the very beginning.