Every time you open any file in your system, it is stored in your system's cache memory. Cache memory is very fast memory that is used in our system to improve the performance of system by giving a faster access to recent accessed file to smooth our interaction and navigation among folder faster.
Likewise when you connect your flash drive or any USB cable device with your computer system data that is shared between your computer's Hard Disk Drive and USB device is cached to give a better performance. When you are working with the files in your device directly then modification that you made to files are not saved but rather cached. So if you remove the Flash drive from your system directly then these updates will be lost from your file that you have updated.
When you safely remove your device from your system then all the updates that you have made to your file are saved properly by moving updates from cache memory to flash memory. So modification are permanently stored on flash drive so now you can easily disconnect your device.
Those who want to avoid safely remove :
To avoid the process of safe removal you can turn off your device caching.
Steps to Turn off caching ::
1. Right Click My Computer -> Select Properties.
2. Click Device Manager
3. Now Expand Disk Drive and Select Your HDD(Hard Disk Drive)
4. Click Properties and Select Policy tab
5. Finally select Quick Removal to Avoid messing with Safe Removal.
Now you don't need to safely remove just plug it off.