But its already the case that you cant shrink a column to less than 4 px in width
In my opinion, borders of a column are never indication of the column (especially if they're only 4 tiny pixels apart from each other). It would be easier to see that there's a column, if we could see at least 1 character of the column's title (or something like "..." in it's place). If there's a column, we need to know that it's there at all times. If we would want to hide it, we could easily do it by unchecking it in the right-click dropdown menu.