To study the type of a critical point we look at second derivatives.
fxx = 2
fxy = k
fyy = 8 .
We see that (0,0) is a minimum, when 16 − k2 > 0 or equivalently |k| < 4. When |k| > 4, the point (0, 0) is a saddle point.
What happens for k = ±4? Then the second derivative test is inconclusive and we have to take a closer look at the function. We have
f (x, y) = x2 ± 4xy + 4y2 = (x ± 2y)2≥0 .
Thus (0, 0) is a minimum for k = ±4
