I am familiar with this problem and the simplest solution is to use post Newton maths involving Gauss’s Divergence Theorem which states;
“The gravitational flux through a surface is proportional to the enclosed mass.”
If the hollow and solid spheres are concentric the problem is reduced to Newton’s Shell Theorem which states there is no gravitational force acting on a particle anywhere within the hollow.
Since this is not the case the solution is to use a bit of trickery which appears not to have a physical significance.
This is to consider a sphere with a positive density ρ and sphere with a
negative density –ρ.
The idea is when the spheres are added, the densities cancel each other out and you are left with a hollow.
The rational behind this is that there is a similar problem in electrostatics for a cavity in an electric field where the cavity is the sum of positive and electric charges which are physically real.
Mathematically Gauss’s Divergence Theorem is;
V is the volume bounded by a closed surface S, d
A is a surface element.
Pictorially the theorem for a sphere is depicted as follows (here dS = dA).
For the sphere the equation is easily solved.
The left hand integral is the integration over a surface S and equals g(r)4πr² where 4πr² is the surface area of a sphere.
The right hand integral is the integration over a volume V and equals -4πGρ/3(r³) where 4π/3(r³) is the volume of a sphere.
Hence the solution to Gauss's divergence theorem for a sphere is;
g(r) = -4πGρ/3(
r)
g is the acceleration represented as a gravitational vector field and is a function of the radial vector
r, G is the gravitational constant.
Suppose the sphere with the positive density ρ has a centre at
r =
x₁ and the sphere with the negative density –ρ and centre at
r =
x₂.
Applying Gauss’s divergence Theorem to the positive and negative density spheres results in the gravitational field equations;
g₁(r) = -4πGρ/3(
r-x₁) and
g₂(r) = 4πGρ/3(
r-x₂) respectively.
The condition |r - x₁| > |r - x₂| applies since the positive density sphere is larger with the negative density sphere enclosed within.
The superposition principle allows us to add the gravitational fields as Newtonian gravity is linear.
g(r) =
g₁(r) +
g₂(r) = -4πGρ/3(
x₁-x₂)
g(r) is the gravitational field of the hallow and has the constant value -4πGρ/3.
A particle of mass m in this hallow has a gravitational force -4πGρm/3 acting on it in the direction of the vector (
x₁-x₂).
Note if
x₁ = x₂, the shells are concentric and
g(r) = 0 which is Newton’s Shell Theorem.