The first big difference is, of course, the period of stay. With a Spouse Visa you can stay in Japan up to 5 years with the same visa, while the Permanent Visa has no duration limit. You also don’t have to gather and write various documents for a renewal, nor do you have to pay the 5,000 yen fee for a new visa. The only thing you still have to change is your Residency Card. However, with a Permanent Visa, you only have to change it every 7 years, unlike the 5 year period if you have a Spouse Visa.
With the Permanent Visa also comes more trust. Banks are more likely to loan you money to buy a house or a car, since having this visa means that you have gained the trust of the Japanese Government. It is also easier to get credit cards, as well as find jobs, or even geta contract with a cell phone company.
All in all, the Permanent Residency Visa will give you a lot more comfort and trust all around. If you intend to stay in Japan forever or for an extended period of time, it may be best to change to a Permanent Visa.