The off-grid Castle changes both in looks and size when you reach certain thresholds as you level it up. The Castle can reach up to level 15. Please, be aware that your Castle System progress is world based!
Level 0 (default)
In case you don't want to share the progress of your Castle with other players, keep in mind you can deactivate the Castle visuals through the game settings, meaning it will always show the level 1 graphics for visitors.