Am I learning with AI or just knowing how to use it?

Am I learning with AI… or just knowing how to use it?
By: Walid Marwan

By chance, a recently published paper in Nature Human Behaviour appeared before me.
The paper’s title was:
Generative AI and the temporary upskilling of knowledge workers
The title caught my eye, but what made me keep reading was the question that followed: when artificial intelligence helps me do something I didn’t know how to do before, does that mean I have actually learned the skill? Or am I merely able to perform it as long as the tool is available?
The question stayed with me: am I learning with AI… or just knowing how to use it?

A study made me pause.

The researchers conducted an experiment with 487 management consultants, asking them to perform data-science tasks beyond their expertise. One group was given access to ChatGPT and trained on its use, while another group completed the tasks without it.
The expected result occurred: the people who used artificial intelligence performed better on the tasks.
So far, there’s no big surprise. But the part that truly caught my attention came next.
When the participants’ technical knowledge was tested after the experiment without using ChatGPT, the researchers found no evidence that the group that used artificial intelligence had acquired more technical knowledge than the other group.

In other words: the tool boosted their performance while it was present, but the performance gain did not necessarily mean that learning had occurred.
The researchers used a metaphor I liked a lot, describing artificial intelligence as an external scaffold for skills; something that raises your capability while you use it, but the extra capability may depend on its continued presence.
That made me think about my work in learning and development differently.

Performance is not learning

We sometimes conflate two very close things: a person performing better and a person actually becoming better.
If an employee uses artificial intelligence and writes a better report, that is an improvement in performance. If they manage to analyze data they previously could not, that is also an improvement in performance.
But will they, after a while, be able to do part of that on their own? Did they understand the logic? Did they acquire the way of thinking? And if the tool disappears, will any of the skill remain?
Here the distinction between completing a task and actual learning begins.
That is a very important distinction for me. In learning and development, our goal is not only to help people do their work today faster. We also aim to build their capabilities so they can tackle tougher challenges tomorrow.

I have also started noticing this in myself

The truth is that the issue is not limited to employees or learners. I began asking myself the same question.

• When I need an idea, I can ask artificial intelligence.
• When I need to write something, it can help me.
• When I need to analyze a topic, I can request different perspectives from it within seconds.

That saved me a lot of time and raised the quality of a large part of my work.

But sometimes I have to ask myself: am I using the tool to expand my thinking… or to relieve me of thinking?
The difference between them is subtle in form, but very large in outcome.
For example, there is a difference between telling artificial intelligence, “Solve this problem for me.”
and between thinking first, then saying to it, “This is the solution I arrived at. What might I be missing?”
In the first case, I’m requesting an answer. In the second, I’m using it to challenge my way of thinking.
Both can lead to a good result, but the learning journey is entirely different.

Perhaps we need to add a new question for L&D

With the spread of artificial intelligence, it is natural to focus on teaching people how to use it better. That is important.
But there may be another question that should be present in the design of any learning experience: does the use of artificial intelligence here help the learner build the skill… or only help them complete the task?
Because the two are not the same thing.

Sometimes it may be best for a person to try first, then use artificial intelligence for comparison. Or ask it to critique the solution instead of giving the solution. Or read the answer and then explain it in their own words. Or choose among multiple suggestions and justify the choice.
The idea is not to make using artificial intelligence harder, nor to revert to doing everything the old way. The idea is to design a usage method that makes the human think together with the tool, not stop thinking because of it.

Speed is important… but it is not the only goal

One of the biggest advantages of artificial intelligence is that it shortens tasks that used to take a long time. Of course it is illogical to ignore this advantage.
But the problem starts when our only success metric is: “How much faster did we finish?”
We may also need to ask: “What did we learn while we were finishing?”
Because the fastest way to finish a task is not always the best way to build a skill.
And sometimes a small part of trying, erring, thinking, and retrying is exactly the part that creates learning.

In the end

The paper that happened upon me by chance did not make me any less enthusiastic about artificial intelligence. On the contrary.
It made me think more about how to use it.
I don’t want artificial intelligence only to make me do more or faster.
I want it to help me understand more, think better, and learn things that stay with me even after I close the screen.
Perhaps the real test of learning in the AI era is not: “What could you accomplish using AI?”
But: “What do you have after AI is no longer present?”
Because if the tool made me do something better, my performance improved.
But if I leave the experiment understanding, thinking, and capable more than before… then I truly have learned.
________________________________________
Source:
Wiles, E., Krayer, L., Abbadi, M. et al. (2026). Generative AI and the temporary upskilling of knowledge workers. Nature Human Behaviour.
DOI: 10.1038/s41562-026-02566-6

About the author: Walid Marwan, talent development strategist, learning and development director, trainer, facilitator, and certified career coach, with over 18 years of professional experience across various sectors. He currently serves as Learning and Development Director at Tamr Group in Saudi Arabia, leading the design and implementation of talent development strategies, leadership programs, corporate learning, competency-based development, and building a coaching culture within organizations. He also serves as board member and events and community director for the Saudi chapter of the International Coaching Federation (ICF), is a founding partner in the community The Career Hive, and co-author of the book “Al-Kholasa”. His professional approach centers on learning, empowerment, and developing individuals from potential to performance.