Cook diced bacon in a large pot or Dutch oven over medium heat until crispy, then remove bacon with a slotted spoon and set aside, reserving 2 tablespoons of bacon grease in the pot.

Add diced onion and celery to the pot with bacon grease and cook until softened, about 5-7 minutes.

Stir in diced potatoes and flour, cooking for 2 minutes, then gradually whisk in clam juice and milk until smooth.

Bring the mixture to a simmer, then reduce heat and cook until potatoes are tender and the soup has thickened, about 15-20 minutes.

Stir in heavy cream, drained clams, salt, and pepper, heating gently until warmed through without boiling.