This file derives the static solution (see output following (*)) and ODEs (see output following (**) for all original ODEs, or following (***) for the core system after a change of variables) for general payoff specifications. 
To expand fully, comment out commands that set u[dot] and uh[dot] terms to 0.

SetOptions[EvaluationNotebook[],CellContext->Notebook]
ClearAll[Evaluate[Context[]<>"*"]]
ClearAll[Derivative]
ν=0;

Guess the value function V (θ, m, l, t) where m is second order belief

V=v0[t]+v1[t]θ+v2[t]m+v3[t]l+v4[t](θ^2)+v5[t](m^2) +v6[t](l^2)+v7[t]θ m+v8[t]θ l+v9[t]m l;

Long-run player’s flow payoff ((x^T)Λ x,where x=(a,ahat,θ,1)); note that uaa,uahah,uθθ,uaah,uaθ and uahθ are precisely the corresponding second derivatives or mixed partials. Payoffs have been scaled so that U_{aa}=-1. To complete the Λ matrix, we use the notation u[dot]c for coefficients on linear terms (in a, ah=ahat, or θ) and uc for the constant term.

(*uθθ=0;*)
uc=0;
(*uac=0;*)
uahc=0;
uθc=0;
uahθ=0;
 uaa=-1 ;(*Assumption 1, part (i)*)
Λ={{ uaa/2,uaah,uaθ,uac},{0,uahah/2,uahθ,uahc},{0,0,uθθ/2,uθc},{0,0,0,uc}};
MatrixForm[Λ]

Similarly, define the short-lived player’s payoff ((x^T)Λh x,where x=(a,ahat,θ,1)). Scale so that Uh_{ahah}=-1.

uhθθ=0;
uhc=0;
uhac=0;
(*uhahc=0;*)
uhθc=0;
uhaθ=0;
uhaa=0;
uhahah=-1;(*Assumption 1, part (i)*)
Λh={{uhaa/2,uhaah,uhaθ,uhac},{0,uhahah/2,uhahθ,uhahc},{0,0,uhθθ/2,uhθc},{0,0,0,uhc}};
MatrixForm[Λh]

Guess the strategy of long-lived player, define shorthand for later use

a=β0[t]+β1[t]m+β2[t]l+β3[t]θ
α0[t]=β0[t];
α2[t]=β2[t]+β1[t](1-χ[t]);
α3[t]=β3[t]+β1[t]χ[t];
γ2[t]=χ[t]γ[t];

Expectation of long-lived player’s action from perspective of myopic player, using A for α to prevent substitution for purpose of display

aexp=A0[t]+l A2[t]+m A3[t]

Long-lived player' s expectation of short-lived player's myopic best response

aharg=ArgMax[Simplify[x.Λh.x/.x->{adummy,ahdummy,θ,1}],ahdummy]/.{adummy->aexp,θ->m}
δ0[t]=aharg/.{m->0,l->0}/.{A0[t]->α0[t],A2[t]->α2[t],A3[t]->α3[t]};
δ1[t]=Coefficient[aharg,m]/.{A0[t]->α0[t],A2[t]->α2[t],A3[t]->α3[t]};
δ2[t]=Coefficient[aharg,l]/.{A0[t]->α0[t],A2[t]->α2[t],A3[t]->α3[t]};
ah=δ0[t]+δ1[t]m+δ2[t]l

Define Various Constants

Σ=(ν^2)/((σX)^2)+1/((σY)^2);
μ1[t]=α3[t]γ[t]Σ;
μ0[t]=-μ1[t](α0[t]+α2[t]l);
μ2[t]=-α3[t]μ1[t];
Bh[t]=(ν α3[t]γ[t]+γ2[t]δ1[t])/((σX)^2);
μh0[t]=-Bh[t](ν β0[t]+δ0[t]);
μh1[t]=α3[t]γ[t](β1[t]-α3[t])Σ-Bh[t](ν β1[t]+δ1[t]);
μh2[t]=α3[t]γ[t](β2[t]-α2[t])Σ-Bh[t](ν β2[t]+δ2[t]);
μh3[t]=α3[t]γ[t]β3[t]Σ-Bh[t]ν β3[t];
μL[t]=FullSimplify[(μh1[t]+μh2[t]+μh3[t])l/(1-χ[t])+(μh0[t]+Bh[t](ν x+δ0[t]+δ1[t]m+δ2[t]l))/(1-χ[t])];
μM[t]=μ0[t]+μ2[t]m+μ1[t]x;
σL[t]=Bh[t]σX/(1-χ[t]);
σM[t]=Bh[t]σX;

Player 2' s objective

xvec={x,ah,θ,1};
OBJ=xvec.Λ.xvec+uahah/2 ((δ1[t])^2)γ[t]χ[t]+D[V,t]+μM[t]D[V,m]+μL[t]D[V,l]+(((σM[t])^2)/2)D[V,{m,2}]+σM[t]σL[t]D[V,m,l]+(((σL[t])^2)/2)D[V,{l,2}];

First-order condition

FOC=D[OBJ,x]/.x->a;

Extract constant term

cons1=FullSimplify[FOC-θ D[FOC,θ]-m D[FOC,m]-l D[FOC,l]];

Match coefficients to obtain static solutions (β0st,β1st,β2st,β3st). Observe that these are well-defined under Assumption 1, part (iv).

sol1=Simplify[{v2[t],v5[t],v7[t],v9[t]}/.Solve[{Coefficient[FOC,m]==0,Coefficient[FOC,θ]==0,Coefficient[FOC,l]==0,cons1==0},{v2[t],v5[t],v7[t],v9[t]}][[1]]];
v2s[t]=sol1[[1]]
v5s[t]=sol1[[2]]
v7s[t]=sol1[[3]]
v9s[t]=sol1[[4]]
solst={β0[t],β1[t],β2[t],β3[t]}/.FullSimplify[Solve[{v2s[t]==0,v5s[t]==0,v7s[t]==0,v9s[t]==0},{β0[t],β1[t],β2[t],β3[t]}][[1]]];
β0st=solst[[1]];
β1st=solst[[2]];
β2st=solst[[3]];
β3st=solst[[4]];

(*) Static solution

Style[{β0st,β1st,β2st,β3st},Background->Yellow]

Total Signaling Coefficient: observe that it is positive under Assumption 1, part (ii).

Simplify[β1st χ[t]+β3st]

Differentiate coefficients

v2p[t]=D[v2s[t],t];
v5p[t]=D[v5s[t],t];
v7p[t]=D[v7s[t],t];
v9p[t]=D[v9s[t],t];

Learning ODEs (forward system)

γd[t]=FullSimplify[-((γ[t])^2)((α3[t])^2)Σ/.α3[t]->α[t]]
χd[t]=FullSimplify[γ[t] ((α3[t])^2)Σ(1-χ[t])-(ν α3[t]+δ1[t]χ[t])Bh[t]/.α3[t]->α[t]]

Now solve the HJB by matching terms.

HJB=(-r V+OBJ)/.x->a;
HJB1=Simplify[D[HJB,θ]/.{m->0,l->0,θ->0,v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB2=Simplify[D[HJB,m]/.{m->0,l->0,θ->0,v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB3=Simplify[D[HJB,l]/.{m->0,l->0,θ->0,v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB4=(1/2)Simplify[D[HJB,{θ,2}]/.{v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB5=(1/2)Simplify[D[HJB,{m,2}]/.{v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB6=(1/2)Simplify[D[HJB,{l,2}]/.{v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB7=Simplify[D[HJB,m,θ]/.{v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB8=Simplify[D[HJB,l,θ]/.{v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB9=Simplify[D[HJB,m,l]/.{v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJB0=Simplify[HJB/.{m->0,l->0,θ->0,v2[t]->v2s[t],v2'[t]->v2p[t],v5[t]->v5s[t],v5'[t]->v5p[t],v7[t]->v7s[t],v7'[t]->v7p[t],v9[t]->v9s[t],v9'[t]->v9p[t]}/.{γ'[t]->γd[t],χ'[t]->χd[t]}];
HJBsys={v0'[t],v1'[t],v3'[t],v4'[t],v6'[t],v8'[t],β0'[t],β1'[t],β2'[t],β3'[t]}/.Simplify[Solve[{HJB0==0,HJB1==0,HJB2==0,HJB3==0,HJB4==0,HJB5==0,HJB6==0,HJB7==0,HJB8==0,HJB9==0}/.{χ'[t]->χd[t],γ'[t]->γd[t]},{v0'[t],v1'[t],v3'[t],v4'[t],v6'[t],v8'[t],β0'[t],β1'[t],β2'[t],β3'[t]}][[1]]/.β3[t]->α[t]-β1[t]χ[t]];
v0d[t]=Simplify[HJBsys[[1]]/.β3[t]->α[t]-β1[t]χ[t]];
v1d[t]=Simplify[HJBsys[[2]]/.β3[t]->α[t]-β1[t]χ[t]];
v3d[t]=Simplify[HJBsys[[3]]/.β3[t]->α[t]-β1[t]χ[t]];
v4d[t]=Simplify[HJBsys[[4]]/.β3[t]->α[t]-β1[t]χ[t]];
v6d[t]=Simplify[HJBsys[[5]]/.β3[t]->α[t]-β1[t]χ[t]];
v8d[t]=Simplify[HJBsys[[6]]/.β3[t]->α[t]-β1[t]χ[t]];
β0d[t]=Simplify[HJBsys[[7]]/.β3[t]->α[t]-β1[t]χ[t]];
β1d[t]=Simplify[HJBsys[[8]]/.β3[t]->α[t]-β1[t]χ[t]];
β2d[t]=Simplify[HJBsys[[9]]/.β3[t]->α[t]-β1[t]χ[t]];
β3d[t]=Simplify[HJBsys[[10]]/.β3[t]->α[t]-β1[t]χ[t]];
α3d[t]=Simplify[β1d[t]χ[t]+β1[t]χd[t]+β3d[t]/.β3[t]->α[t]-β1[t]χ[t]];

(**) Forward System ODEs: observe that the core ODEs (v6,v8,β1,β2,β3) are well-defined given Assumption 1. Observe also that α factors out of its ODE. Given that its terminal value is nonzero (Assumption 1, part (ii)), the comparison theorem implies that any solution is nonzero.

v6d[t]
v8d[t]
β1d[t]
β2d[t]
β3d[t]
α3d[t]

(**) Remaining ODEs; these are also well-defined given α != 0.

β0d[t]
v0d[t]
v1d[t]
v3d[t]
v4d[t]

(***) Change of variables to eliminate (1 - χ[t]) from denominators of v6’,v8’,b1’,b2’,b3’

Clear[v6new,v8new,β2new]
v6NEW[t]=v6[t]γ[t]/(((1-χ[t]))^2);
v8NEW[t]=v8[t]γ[t]/(1-χ[t]);
β2NEW[t]=β2[t]/(1-χ[t]);
(*FORWARD SYSTEM ODES*)
v6NEWd[t]=FullSimplify[D[v6NEW[t],t]/.{v6'[t]->v6d[t],v8'[t]->v8d[t],α'[t]->α3d[t],β1'[t]->β1d[t],β2'[t]->β2d[t],β3'[t]->β3d[t],γ'[t]->γd[t],χ'[t]->χd[t]}/.{v6[t]->v6new[t](((1-χ[t]))^2)/γ[t]}/.β2[t]->β2new[t](1-χ[t])]
v8NEWd[t]=FullSimplify[D[v8NEW[t],t]/.{v6'[t]->v6d[t],v8'[t]->v8d[t],α'[t]->α3d[t],β1'[t]->β1d[t],β2'[t]->β2d[t],β3'[t]->β3d[t],γ'[t]->γd[t],χ'[t]->χd[t]}/.{v6[t]->v6new[t](((1-χ[t]))^2)/γ[t]}/.β2[t]->β2new[t](1-χ[t])/.{v8[t]->v8new[t](1-χ[t])/γ[t]}]
β1NEWd[t]=
FullSimplify[β1d[t]/.{v6'[t]->v6d[t],v8'[t]->v8d[t],α'[t]->α3d[t],β1'[t]->β1d[t],β2'[t]->β2d[t],β3'[t]->β3d[t],γ'[t]->γd[t],χ'[t]->χd[t]}/.{v6[t]->v6new[t](((1-χ[t]))^2)/γ[t]}/.β2[t]->β2new[t](1-χ[t])/.{v8[t]->v8new[t](1-χ[t])/γ[t]}]
β2NEWd[t]=Simplify[D[β2NEW[t],t]/.{v6'[t]->v6d[t],v8'[t]->v8d[t],α'[t]->α3d[t],β1'[t]->β1d[t],β2'[t]->β2d[t],β3'[t]->β3d[t],γ'[t]->γd[t],χ'[t]->χd[t]}/.{v6[t]->v6new[t](((1-χ[t]))^2)/γ[t]}/.β2[t]->β2new[t](1-χ[t])/.{v8[t]->v8new[t](1-χ[t])/γ[t]}]
β3NEWd[t]=Simplify[β3d[t]/.{v6'[t]->v6d[t],v8'[t]->v8d[t],α'[t]->α3d[t],β1'[t]->β1d[t],β2'[t]->β2d[t],β3'[t]->β3d[t],γ'[t]->γd[t],χ'[t]->χd[t]}/.{v6[t]->v6new[t](((1-χ[t]))^2)/γ[t]}/.β2[t]->β2new[t](1-χ[t])/.{v8[t]->v8new[t](1-χ[t])/γ[t]}]
α3NEWd[t]=Simplify[α3d[t]/.{v6'[t]->v6d[t],v8'[t]->v8d[t],α'[t]->α3d[t],β1'[t]->β1d[t],β2'[t]->β2d[t],β3'[t]->β3d[t],γ'[t]->γd[t],χ'[t]->χd[t]}/.{v6[t]->v6new[t](((1-χ[t]))^2)/γ[t]}/.β2[t]->β2new[t](1-χ[t])/.{v8[t]->v8new[t](1-χ[t])/γ[t]}]

Example: check β1 ODE (after change of variables) for Leading by Example game, multiplying U by a factor of 1/4 so that U_{aa}=1. The only difference is that value function coefficients here are 1/4 of corresponding value function coefficients in Theorem 2, due to the rescaling.

FullSimplify[β1NEWd[t]/.{r->0,uaθ->2/4,uaah->2/4,uahah->-2/4,uhaah->1}]