$(D^2+2)y=x^2e^{3x}+e^x-\cos 2x \ $ Auxillary equation is $D^2+2=0 \\ \therefore D= \pm\sqrt{2i} \\ \therefore C.f \space is \\ $ $y_c=e^{Dx}(C_1\cos\sqrt2x+C_2\sin \sqrt2 x) \ $ Now particular integral is $y_p=\dfrac 1{f(D)} \ $ $\therefore y_p=\dfrac 1{D^2+2}[x^2e^{3x}+e^x-\cos 2x] \ =\dfrac 1{D^2+2}x^2e^{3x} + \dfrac 1{D^2+2}e^x-\dfrac 1{D^2+2}\cos 2x\ $ $=e^{3x}\dfrac 1{(D+3)^2+2}x^2+e^x\dfrac 1{1^2+2}-\cos 2x\dfrac 1{-(2)^2+2} \ $ $=e^{3x}\dfrac 1{D^2+6D+11}x^2+\dfrac {e^x}3+\dfrac {\cos 2x}2\ $ $=\dfrac {e^x}3+\dfrac {\cos 2x}2+\dfrac {e^{3x}}{11}\Bigg[1+\Bigg(\dfrac {D^2+6D}{11}\Bigg)\Bigg]^{-1}x^2\ $ $=\dfrac {e^x}3+\dfrac {\cos 2x}2+\dfrac {e^{3x}}{11}\Bigg[1-\Bigg(\dfrac {D^2+6D}{11}\Bigg)+\Bigg(\dfrac {D^2+6D}{11}\Bigg)^2\Bigg]x^2\ $ $=\dfrac {e^x}3+\dfrac {\cos 2x}2+\dfrac {e^{3x}}{11}\Bigg[1-\dfrac {6D}{11}-\dfrac {D^2}{11}+\dfrac {6D^2}{11}+...\Bigg]x^2\ $ $=\dfrac {e^x}3+\dfrac {\cos 2x}2+\dfrac {e^{3x}}{11}\Bigg[1-\dfrac {6D}{11}+\dfrac {25D^2}{11}+...\Bigg]x^2\ $ $=\dfrac {e^x}3+\dfrac {\cos 2x}2+\dfrac {e^{3x}}{11}\Bigg[x^2-\dfrac {12x}{11}+\dfrac {50}{121}\Bigg]\ $ The complete solution is $y_c +y_p\ $ $y=C_1\cos\sqrt2x+C_2\sin\sqrt2 x+\dfrac {e^x}3+\dfrac {\cos 2x}2+\dfrac {e^{3x}}{11}\Bigg[x^2-\dfrac {12x}{11}+\dfrac {50}{121}\Bigg]\ $