Find the factorization of and use it to solve the system

You can earn partial credit on this problem.