ADF Pipeline Still Uses Old Linked Service After Updating Sink Dataset and Publishing

Question

ADF Pipeline Still Uses Old Linked Service After Updating Sink Dataset and Publishing

Suyash Kant 40

Hi,

I'm encountering an issue in Azure Data Factory where my pipeline seems to ignore an updated linked service for the sink dataset.

Here’s the scenario:

I have a pipeline with a Copy Data activity.

The sink dataset was originally using Linked Service A.

I updated the sink dataset to use Linked Service B instead.

I saved and published the changes.

I verified the change in Azure DevOps—specifically in the pipeline and dataset JSON templates—and confirmed that the sink now references Linked Service B.

Despite all this, when I run the pipeline, the data still ends up in the database associated with Linked Service A.

What I've already checked:

The sink dataset is correctly pointing to Linked Service B.

The connection string in Linked Service B points to the new/different database.

I re-published multiple times and verified changes in DevOps to ensure the correct version is deployed.

It seems like the pipeline is still somehow holding on to the old linked service. Has anyone seen this behavior before or know what might be causing it?

Thanks!

Venkat Reddy Navari 2,300 Reputation points Microsoft External Staff Moderator

2025-06-10T10:27:29.31+00:00

@Suyash Kant Just checking in to see if the below answer provided by @Vinodh247 helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Accepted answer

0 additional answers

Your answer

Venkat Reddy Navari 2,300 Reputation points Microsoft External Staff Moderator

2025-06-10T10:27:29.31+00:00

@Suyash Kant Just checking in to see if the below answer provided by @Vinodh247 helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

this is a known behavior in ADF, and there are a few possible reasons why your pipeline is still writing to the original 'Linked Service A' despite updating the sink dataset:

Key Possibilities and Solutions

Sink dataset overridden at activity level

Even though the dataset is updated globally, check if the Copy Data activity itself has the linked service overridden at the activity level.

Open the Copy Data activity.

Go to the Sink tab.

Check if it has a manually selected linked service or if it is pointing to a dataset that is internally configured with Linked Service A.

Resolution: Remove any manual override and ensure the sink is using the dataset referencing Linked Service B.

Parameterised linked service or dataset

Are your datasets or pipelines using parameters to determine the linked service?

If so, the parameter might still be passing a reference to Linked Service A during runtime.
Resolution: Check runtime parameters and make sure the correct linked service or dataset reference is being passed.

Pipeline or dataset caching Issue

Sometimes, ADF UI or runtime caches old references, especially if you are debugging or re-running from the "Debug" button instead of triggering a new run.

Resolution: Do a full trigger run (not debug), or restart your browser/session, clear cache, and try again.

Old artifact deployed in IR

If you are using self-hosted IR or CI/CD pipelines (via Azure DevOps), ensure that the correct ARM template was actually deployed to the target environment.

Resolution: Double-check the Linked Service deployment logs, and consider redeploying just the linked service and datasets manually to confirm.

Copy activity uses Inline sink Instead of dataset

Sometimes, the Copy Activity may have an inline sink (directly configured within the activity) rather than referencing a separate dataset.

Resolution: Open the JSON view of the pipeline and verify whether the sink is referencing a dataset or has inline settings that still use Linked Service A.

Steps to Troubleshoot Further

Open JSON view of the pipeline in ADF Studio and verify the sink section.

Trace the exact dataset JSON to confirm no hidden reference or parameter is reverting to Linked Service A.

Run the pipeline with verbose logging and inspect logs to confirm which linked service is being used.

If the issue persists, export the pipeline + dataset JSONs and search for LinkedServiceA to confirm no lingering references.

Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

Suyash Kant 40 Reputation points

2025-06-10T11:13:44.96+00:00

So I check the live mode of my data factory and there the dataset is still pointing to the old linked service. Essentially the linked service for the dataset in the git collaboration branch and publish branch is correct but it is still pointing to the old linked service in my data factory's live mode. Could this be the root cause, if yes then how can i solve it?
Venkat Reddy Navari 2,300 Reputation points Microsoft External Staff Moderator

2025-06-12T09:53:13.7966667+00:00

Suyash Kant In Git-integrated ADF, it’s important to remember that the changes made in your collaboration or publish branches don’t reflect in the live mode until you explicitly publish them. So even if everything looks correct in Git and DevOps, the live factory can still be running on old configurations if a full publish hasn't taken place.

Here’s what you can try:

Go to ADF Studio and click Publish All to push your latest changes to the live environment.

After publishing, switch to Live Mode and check the dataset again to make sure it now points to the correct linked service (Linked Service B).

If it still shows the old one:

Try making a small change and then undoing it to trigger the Publish All button.

Re-publish and refresh the ADF portal (or open it in a private/incognito window to avoid UI cache issues).

As a backup, you can also export the ARM templates from your Git branch and manually deploy them, just to be 100% sure the live environment matches your intended configuration.

This type of mismatch is a common gotcha with Git mode in ADF. Once you republish and confirm in live mode, the pipeline should pick up the correct linked service.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

ADF Pipeline Still Uses Old Linked Service After Updating Sink Dataset and Publishing

0 additional answers

Your answer