TL;DR: Terraform provisions infrastructure and tracks it in state, so it knows what exists and can destroy it. Ansible configures existing machines and holds no state, so it converges a system toward a described condition every run. The failure comes from using Terraform provisioners to configure, or Ansible to provision, when the chosen modules and workflow do not provide the lifecycle control you need.
How to approach it
Give the split in terms of state ownership rather than task type, since that is the real difference. Then give the concrete failure, because the question asks for one.
A strong answer
Terraform is declarative with state. It records a mapping from your configuration to real resource IDs, which is what lets it compute a diff, destroy what you removed, and detect drift. Providers support both in-place changes and replacement; the plan exposes which is required. Terraform can manage immutable compute and mutable managed services.
Ansible is declarative per task with no state file. Each module describes a desired condition (this package installed, this file present with these contents, this service running) and checks before acting, so running it twice changes nothing the second time. Because it holds no state, it cannot tell you what it created and cannot remove what you deleted from a playbook: taking a task out of a playbook does not uninstall the package.
That is the whole distinction. Terraform knows what should exist. Ansible knows what a machine should look like.
The failure from using Terraform to configure. Provisioners (remote-exec, local-exec) run a script at creation time and their effect is invisible to state. Terraform cannot tell whether the script succeeded in a way that persists, cannot detect that someone changed the result afterwards, and does not automatically reconverge it on every apply. A separate terraform_data replacement can rerun a provisioner without replacing the infrastructure, but still does not manage the script's effects as resource attributes. So you get a resource that Terraform believes is correct and is not, and a plan that shows no changes. HashiCorp documents provisioners as a last resort for exactly this reason. The right answer is to bake the configuration into an image (Packer) or use cloud-init, so the desired state is part of what Terraform creates.
The failure from using Ansible to provision. Ansible's cloud modules can create instances, and people do it. Cloud modules can query provider APIs, identify existing instances and support state: absent for deletion. Their behavior depends on the module and identity parameters. The missing facility is a Terraform-style persistent graph that plans removal merely because a declaration disappeared; design explicit inventory and teardown rather than assuming duplicates are inevitable.
The clean division in practice: Terraform creates the network, the instances, the managed database, the load balancer and the DNS. If the instances need configuration, prefer an image built by Packer. Where you have long-lived machines to manage (which is common in enterprises and rare in cloud-native shops), Ansible handles their ongoing configuration, driven from an inventory that can be generated dynamically from the cloud provider.
The honest modern note: on a container platform this question mostly dissolves, because the configuration is the image and there are no long-lived machines to converge. It stays very relevant on hybrid and on-premises estates, which is why it is asked most in enterprise and services interviews.
What interviewers probe next
"What does idempotent mean for an Ansible module?" It checks current condition before acting and reports changed only when it modified something. A command task is not idempotent unless its operation converges safely or you add an appropriate guard such as creates. changed_when only controls reporting, which is the most common way playbooks lie about having changed nothing.
"How would you configure an instance Terraform just created, properly?" Pre-baked image first, cloud-init second, a configuration tool triggered outside Terraform third. Provisioners last.
"Is Ansible's lack of state a weakness?" It is a trade. No state means nothing to corrupt, lock or lose, and cleanup must be explicitly declared rather than inferred from removed tasks.
Common mistakes
Answering only "provisioning versus configuration management", which is the textbook line and does not show you have hit the edges.
Defending Terraform provisioners as normal practice.
Claiming Ansible is idempotent by nature. Module support varies; inspect the documented action semantics and test convergence, including command and shell tasks.
Sources: Ansible EC2 lifecycle module, Terraform provisioners.