Things started clicking when you shoed the MSB sign bit and I remembered a book from college that talked about this, I managed to find the book.

I was in the section signed numbers of "Digital Fundementals"

Thanks for pointing me in the right direction