Skip to main content

Insufficient Memory on Android CI Server

Insufficient Memory on Android CI Server

We recently faced an error where jenkins in our Android CI setup was aborting builds because of insufficient space on the box. This was strange since we have allocated a huge 250G box solely for this purpose.

While trying to solve this issue, I came across some useful things that I think might be useful for other people and whoever is facing this issue, it’d be worthwhile to check these points:

  • If you have been using gradle’s build cache, check it’s folder’s size ( .gradle folder in your home). In my case, it was taking 123GB !. This is because it never used to clear the cache.

Keep in mind that clearing all of it will slow down your builds. So you’ll have to clean it smartly. One option is going with time based approach. This tip (shared by Gautam Korlam) cleans up files not accessed within the last month:

find ~/.gradle -type f -atime +30 -delete
find ~/.gradle -type d -mindepth 1 -empty -delete

In my case, I readjusted it to be 7 days, instead of 30.
Also, gradle auto-clears this in newer versions (starting from 4.10.1). So updating your gradle would ensure that you won’t have to do it manually again.

  • If you use crashlytics, check .crashlytics folder in your home directory. In my case, it was taking 40GB. I still don’t know why it was taking too much time but you can safely delete this directory as stated here.
    I haven’t gotten any update from those guys so for now, I think I’ll have to clean it manually. At the time of writing this, it’s again occupying 2.5G.

  • Check the workspace folders in each of your job’s home directory. If you have not been performing clean before/after each build, these directories would be huge.
    Delete your build directories and add this step in your jenkins job. This can be added in the Source Code Management section of the job settings.

  • Also, jenkins provides a ‘Discard old build’ option in job settings where you can configure things like : no of days to keep builds, max no of builds to keep, no of days to keep artifacts etc. Do use them.

Some screenshots of this excercise :

  1. See the ‘/’ partition occupying 99% :

242 GB occupied out of 259GB


2. Distribution of that ~250G memory :

Distribution


3. One of our job's folder occupying 13G. You can see that workspace folders are the culprit here.

job directory


4. Final state after performing all the steps mentioned above. 198G of free memory :D

final state

Labels:

Comments

Post a Comment

Popular posts from this blog

Android : AbsSavedState cannot be cast to $SavedState

Android AbsSavedState cannot be cast to $SavedState I came across a strange crash today. This is the stacktrace : Fatal Exception: java.lang.ClassCastException: android.view.AbsSavedState$1 cannot be cast to android.widget.ScrollView$SavedState at android.widget.ScrollView.onRestoreInstanceState(ScrollView.java:1810) at android.view.View.dispatchRestoreInstanceState(View.java:14768) at android.view.ViewGroup.dispatchRestoreInstanceState(ViewGroup.java:3123) at android.view.View.restoreHierarchyState(View.java:14746) at android.support.v4.app.Fragment.restoreViewState(SourceFile:470) at android.support.v4.app.FragmentManagerImpl.moveToState(SourceFile:1326) at android.support.v4.app.FragmentManagerImpl.moveFragmentsToInvisible(SourceFile:2323) at android.support.v4.app.FragmentManagerImpl.executeOpsTogether(SourceFile:2136) at android.support.v4.app.FragmentManagerImpl.optimizeAndExecuteOps(SourceFile:2092) at androi...
2 comments
Read more

Android Tip : Handling back button in Fragments

Android Tip : Handling hardware back button in Fragment and DialogFragment This post explains how to handle hardware back button in a Fragment OR DialogFragment . In DialogFragment, it’s quiet straight forward to achieve this. You’ve to get the dialog instance and set onKeyListener on it : if (getDialog() != null ) { getDialog().setOnKeyListener( new DialogInterface.OnKeyListener() { @Override public boolean onKey (DialogInterface dialog, int keyCode, KeyEvent event) { if (event.getAction() == KeyEvent.ACTION_UP && keyCode == KeyEvent.KEYCODE_BACK) { Timber.i( "hardware back button pressed" ); } } }); } This can be done in the onViewCreated callback. For fragments, this method doesn’t work and fragments doesn’t have a direct callback to catch this event. So in this case, the approach that we follow is : You...
Post a Comment
Read more

Android Material Showcase View - Part 1

In this series, I'll be talking about a library which is used by a lot of android developers for showcasing their in-app content,  MaterialShowcaseView . I used this library sometime back for my work and had to modify it to fit my needs. In this process, I ended up digging it a lot and would like to share what I learned. The original library offers a fix set of features, which are demonstrated by the screenshots in the README. Let's jump on the technicalities right away. The original library offers two things : 1) highlighting a view ( see the README to know what I mean by highlighting here) 2) showing a content box that tells user what the highlighted view is about. Here's how it does this : The library adds a direct view child ( this class ) in the window decor view, so that it's drawn on top of your activity's view ( here ). Then, for drawing the overlay and highlighting our view, it overrides the `onDraw()` method of this view and uses android...
Post a Comment
Read more