Mine does as well. A friend of mine who used to rebuild trans says it's probably worn synchromesh. When I gear up I double clutch and it usually gets past the grinding. On the way down sometimes it won't even go into 3rd. At some point I'm just going to bite the bullet and pull the sucker and do a rebuild.
This is a hottly debated topic on the Spicer transmissions. A lot of gl5 is yellow metal safe. The transmission was designed for gl1 and my trans shifts nice with it which is why I use it. All gl1 is safe. I don't know much about the gl4 so I can't comment on it. Good luck.