Third time deployment start failing

I don’t know what’s going on on the tech side of Cleavr but this is the third time that my deployments started failing.

I had to ssh into my EC2 instance to get pm2 logs since I couldn’t get via the web interface (“There were an error fetching logs”). Once Im over there I get: “Error: Cannot find module ‘/home/cleavr/staging.codotto.com/current/index.js’”.

The issue was solved by modifying my cwd key in pm2 config file.

This is the third time this happens out of nowhere. Last time it was due to a deployment done from Cleavr’s side. Once again, I was able to catch this in staging and not production. Imagine if it was in production. Hours trying to understand why things were not working.

The reason why we pay for this service is to not to have to worry about servers. It makes it hard for us to keep justifying paying for a service that for the third time alters some configuration in our pm2 logs and completely crashes our app.

Hello @bfrancisco,

We’re sorry about the inconveniences caused.

You can find the PM2 Logs from Webapp > Deployments > Select the Deployment > Click on Load PM2 Logs:

Or from Server > Logs > Select PM2 as Log Type > Click on Load

Did you mean that the PM2 Ecosystem was altered in some way? If that’s the case let us know and we’ll investigate the matter in detail.

@anish to explain a bit what happened and the timeline of events.

  1. I don’t touch configurations in cleavr. Configured it once with the help of a tutorial and never touched it again
  2. Cleavr released a feature that broke my staging environment a couple of months ago (Had too many unstable restarts PM2 - #4 by bfrancisco)
  3. I fixed the issue with the help of @ashok
  4. Once again, I never touched any configuration
  5. I deploy my app to staging and things stop working again (what solved my issue was doing the same fix I did the last time - Had too many unstable restarts PM2 - #4 by bfrancisco)
  6. I never touched any configuration
  7. I deploy my staging app and things stop working once again. Same issue as before. I fix it using the same solution as the last time

The only one with access to my account is myself only. There are two options here:

  1. Cleavr deployed another featured/bug fix that broke my pm2 configuration again
  2. Someone altered the contents of my pm2 configuration without my knowledge

Please, if I can help in any other way, let me know. This is an issue that we can’t keep having. Fortunately I was able to find this in staging before going to production, but one day I might miss it and deploy a broken pm2 configuration to production.

Hello @bfrancisco,

Sorry about the inconvenience. We’ll look into the issue and get back to you.

Hello @bfrancisco,

For this issue with PM2 configs changing, is this all occurring for the same web app or are there more one web app involved? Such as a staging and a production app? Just looking to rule that out as configs changed for one web app won’t update configs for a separate web app.

We’re not quite sure what may have happened. We’ve reviewed related code and we don’t have anything that would overwrite existing web app PM2 configs outside of editing from web app > build > pm2 config where we have logic to call the config from the server and then write it back on save…

Can you provide detail on how the app does not work anymore after a deployment - such as, do you see an error, does nothing render, etc? Also, what do you do to resolve it? Redeploy, reconfigure PM2 eco, restart PM2, etc? Some web apps may take a little longer to fire up if PM2 needs to restart during deployment - Strapi for instance can take a couple seconds.

If it happens again on staging and you can afford some time, please send us a support request so we can take a closer look.

@anish @amiedema Thank you for the feedback.

Since the issue happened a few days ago I don’t remember important details.

If this happens again, I will write a detailed report of what happened and all my discoveries to help you guys debug this.

2 Likes