I have seen Monorepo deployment best practices & How to deploy a monorepo - Cleavr docs
But I still don’t get next and strapi running reliably.
They have been running both until I had to deploy an update for next.js. Then strapi stopped running. After re-deploying strapi, next.js stopped working.
Nginx error log
2022/12/13 23:52:32 [error] 1070997#1070997: *118 no live upstreams while connecting to upstream, client: 149.28.172.235, server: staging.example.com, request: “GET / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:52:34 [error] 1070997#1070997: *119 no live upstreams while connecting to upstream, client: 149.28.172.235, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:52:36 [error] 1070997#1070997: *119 no live upstreams while connecting to upstream, client: 149.28.172.235, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:52:38 [error] 1070997#1070997: *119 connect() failed (111: Connection refused) while connecting to upstream, client: 149.28.172.235, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://[::1]:3005/”, host: “staging.example.com”
2022/12/13 23:52:38 [error] 1070997#1070997: *119 connect() failed (111: Connection refused) while connecting to upstream, client: 149.28.172.235, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://127.0.0.1:3005/”, host: “staging.example.com”
2022/12/13 23:52:42 [error] 1070997#1070997: *119 no live upstreams while connecting to upstream, client: 149.28.172.235, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:52:49 [error] 1070997#1070997: *122 connect() failed (111: Connection refused) while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “GET / HTTP/2.0”, upstream: “http://[::1]:3005/”, host: “staging.example.com”
2022/12/13 23:52:49 [error] 1070997#1070997: *122 connect() failed (111: Connection refused) while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “GET / HTTP/2.0”, upstream: “http://127.0.0.1:3005/”, host: “staging.example.com”
2022/12/13 23:52:52 [error] 1070997#1070997: *125 no live upstreams while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:52:53 [error] 1070997#1070997: *125 no live upstreams while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:52:55 [error] 1070997#1070997: *125 no live upstreams while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://localhost/”, host: “staging.example.com”
2022/12/13 23:53:00 [error] 1070997#1070997: *125 connect() failed (111: Connection refused) while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://[::1]:3005/”, host: “staging.example.com”
2022/12/13 23:53:00 [error] 1070997#1070997: *125 connect() failed (111: Connection refused) while connecting to upstream, client: 141.164.37.197, server: staging.example.com, request: “HEAD / HTTP/2.0”, upstream: “http://127.0.0.1:3005/”, host: “staging.example.com”
2022/12/13 23:53:05 [error] 1070997#1070997: *128 connect() failed (111: Connection refused) while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: cms.example.com, request: “GET /admin HTTP/2.0”, upstream: “http://127.0.0.1:9678/admin”, host: “cms.example.com”
2022/12/13 23:53:05 [error] 1070997#1070997: *128 connect() failed (111: Connection refused) while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: cms.example.com, request: “GET /admin HTTP/2.0”, upstream: “http://[::1]:9678/admin”, host: “cms.example.com”
2022/12/13 23:53:07 [error] 1070997#1070997: *131 no live upstreams while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /events/2022-10-24/Optimising_Timetabling HTTP/2.0”, upstream: “http://localhost/events/2022-10-24/Optimising_Timetabling”, host: “staging.example.com”
2022/12/13 23:53:07 [error] 1070997#1070997: *131 no live upstreams while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /favicon.ico HTTP/2.0”, upstream: “http://localhost/favicon.ico”, host: “staging.example.com”, referrer: “https://staging.example.com/events/2022-10-24/Optimising_Timetabling”
2022/12/13 23:55:33 [error] 1070997#1070997: *131 connect() failed (111: Connection refused) while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /events/2022-10-24/Optimising_Timetabling HTTP/2.0”, upstream: “http://[::1]:3005/events/2022-10-24/Optimising_Timetabling”, host: “staging.example.com”
2022/12/13 23:55:33 [error] 1070997#1070997: *131 connect() failed (111: Connection refused) while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /events/2022-10-24/Optimising_Timetabling HTTP/2.0”, upstream: “http://127.0.0.1:3005/events/2022-10-24/Optimising_Timetabling”, host: “staging.example.com”
2022/12/13 23:55:34 [error] 1070997#1070997: *131 no live upstreams while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /favicon.ico HTTP/2.0”, upstream: “http://localhost/favicon.ico”, host: “staging.example.com”, referrer: “https://staging.example.com/events/2022-10-24/Optimising_Timetabling”
2022/12/13 23:58:09 [error] 1070997#1070997: *135 connect() failed (111: Connection refused) while connecting to upstream, client: 88.130.158.192, server: cms.example.com, request: “GET /admin HTTP/2.0”, upstream: “http://127.0.0.1:9678/admin”, host: “cms.example.com”
2022/12/13 23:58:09 [error] 1070997#1070997: *135 connect() failed (111: Connection refused) while connecting to upstream, client: 88.130.158.192, server: cms.example.com, request: “GET /admin HTTP/2.0”, upstream: “http://[::1]:9678/admin”, host: “cms.example.com”
2022/12/13 23:58:11 [error] 1070997#1070997: *131 connect() failed (111: Connection refused) while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /events/2022-10-24/Optimising_Timetabling HTTP/2.0”, upstream: “http://[::1]:3005/events/2022-10-24/Optimising_Timetabling”, host: “staging.example.com”
2022/12/13 23:58:11 [error] 1070997#1070997: *131 connect() failed (111: Connection refused) while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /events/2022-10-24/Optimising_Timetabling HTTP/2.0”, upstream: “http://127.0.0.1:3005/events/2022-10-24/Optimising_Timetabling”, host: “staging.example.com”
2022/12/13 23:58:11 [error] 1070997#1070997: *131 no live upstreams while connecting to upstream, client: 2001:16b8:1e91:da00:a46d:74df:f9d8:a26c, server: staging.example.com, request: “GET /favicon.ico HTTP/2.0”, upstream: “http://localhost/favicon.ico”, host: “staging.example.com”, referrer: “https://staging.example.com/events/2022-10-24/Optimising_Timetabling”
PM2 Log
2022-12-13T23:38:51: PM2 log: App [cms.example.com:3] exited with code [1] via signal [SIGINT]
2022-12-13T23:38:51: PM2 log: App [cms.example.com:3] starting in -cluster mode-
2022-12-13T23:38:51: PM2 log: App [cms.example.com:3] online
2022-12-13T23:38:52: PM2 log: App name:cms.example.com id:2 disconnected
2022-12-13T23:38:52: PM2 log: App [cms.example.com:2] exited with code [1] via signal [SIGINT]
2022-12-13T23:38:52: PM2 log: App [cms.example.com:2] starting in -cluster mode-
2022-12-13T23:38:52: PM2 log: App [cms.example.com:2] online
2022-12-13T23:38:52: PM2 log: App name:cms.example.com id:3 disconnected
2022-12-13T23:38:52: PM2 log: App [cms.example.com:3] exited with code [1] via signal [SIGINT]
2022-12-13T23:38:52: PM2 log: App [cms.example.com:3] starting in -cluster mode-
2022-12-13T23:38:52: PM2 log: App [cms.example.com:3] online
2022-12-13T23:38:54: PM2 log: App name:cms.example.com id:3 disconnected
2022-12-13T23:38:54: PM2 log: App [cms.example.com:3] exited with code [1] via signal [SIGINT]
2022-12-13T23:38:54: PM2 log: App [cms.example.com:3] starting in -cluster mode-
2022-12-13T23:38:54: PM2 log: App [cms.example.com:3] online
2022-12-13T23:38:56: PM2 log: App name:cms.example.com id:3 disconnected
2022-12-13T23:38:56: PM2 log: App [cms.example.com:3] exited with code [1] via signal [SIGINT]
2022-12-13T23:38:56: PM2 log: App [cms.example.com:3] starting in -cluster mode-
2022-12-13T23:38:56: PM2 log: App [cms.example.com:3] online
2022-12-13T23:46:10: PM2 log: App [staging.example.com:4] starting in -cluster mode-
2022-12-13T23:46:10: PM2 log: App [staging.example.com:4] online
2022-12-13T23:46:10: PM2 log: App [staging.example.com:5] starting in -cluster mode-
2022-12-13T23:46:11: PM2 log: App [staging.example.com:5] online
2022-12-13T23:46:11: PM2 log: App [staging.example.com:6] starting in -cluster mode-
2022-12-13T23:46:11: PM2 log: App [staging.example.com:6] online
2022-12-13T23:46:11: PM2 log: App [staging.example.com:7] starting in -cluster mode-
2022-12-13T23:46:11: PM2 log: App [staging.example.com:7] online
2022-12-13T23:47:32: PM2 log: Stopping app:cms.example.com id:0
2022-12-13T23:47:32: PM2 log: Stopping app:cms.example.com id:1
2022-12-13T23:47:32: PM2 error: Error: spawn ps EAGAIN
2022-12-13T23:47:32: PM2 log: ===============================================================================
2022-12-13T23:47:32: PM2 log: — PM2 global error caught ---------------------------------------------------
2022-12-13T23:47:32: PM2 log: Time : Tue Dec 13 2022 23:47:32 GMT+0000 (Koordinierte Weltzeit)
2022-12-13T23:47:32: PM2 error: read ENOTCONN
2022-12-13T23:47:32: PM2 error: Error: read ENOTCONN
at tryReadStart (net.js:586:20)
at Socket._read (net.js:597:5)
at Socket.Readable.read (internal/streams/readable.js:465:10)
at Socket.read (net.js:638:39)
at new Socket (net.js:389:12)
at Object.Socket (net.js:282:41)
at createSocket (internal/child_process.js:322:14)
at ChildProcess.spawn (internal/child_process.js:440:23)
at spawn (child_process.js:677:9)
at spawnWithSignal (child_process.js:911:17)
2022-12-13T23:47:32: PM2 log: ===============================================================================
2022-12-13T23:47:32: PM2 error: [PM2] Resurrecting PM2
events.js:377
throw er; // Unhandled ‘error’ event
^
<ref *1> Error: read ENOTCONN
at tryReadStart (net.js:586:20)
at Socket._read (net.js:597:5)
at Socket.Readable.read (internal/streams/readable.js:465:10)
at Socket.read (net.js:638:39)
at new Socket (net.js:389:12)
at Object.Socket (net.js:282:41)
at createSocket (internal/child_process.js:322:14)
at ChildProcess.spawn (internal/child_process.js:440:23)
at spawn (child_process.js:677:9)
at spawnWithSignal (child_process.js:911:17)
Emitted ‘error’ event on Domain instance at:
at Socket.emit (domain.js:525:12)
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:82:21) {
errno: -107,
code: ‘ENOTCONN’,
syscall: ‘read’,
domainEmitter: Socket {
connecting: false,
_hadError: false,
_parent: null,
_host: null,
_readableState: ReadableState {
objectMode: false,
highWaterMark: 16384,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: [],
flowing: null,
ended: false,
endEmitted: false,
reading: true,
sync: false,
needReadable: true,
emittedReadable: false,
readableListening: false,
resumeScheduled: false,
errorEmitted: true,
emitClose: false,
autoDestroy: false,
destroyed: true,
errored: [Circular *1],
closed: true,
closeEmitted: false,
defaultEncoding: ‘utf8’,
awaitDrainWriters: null,
multiAwaitDrain: false,
readingMore: false,
dataEmitted: false,
decoder: null,
encoding: null,
[Symbol(kPaused)]: null
},
_events: [Object: null prototype] {
end: [Function: onReadableStreamEnd],
close: [Function (anonymous)]
},
_eventsCount: 2,
_maxListeners: undefined,
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: true,
decodeStrings: false,
defaultEncoding: ‘utf8’,
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
buffered: [],
bufferedIndex: 0,
allBuffers: true,
allNoop: true,
pendingcb: 0,
prefinished: false,
errorEmitted: true,
emitClose: false,
autoDestroy: false,
errored: [Circular *1],
closed: true,
writable: false,
closeEmitted: false
},
allowHalfOpen: false,
_sockname: null,
_pendingData: null,
_pendingEncoding: ‘’,
server: null,
_server: null,
[Symbol(async_id_symbol)]: 5991,
[Symbol(kHandle)]: null,
[Symbol(kSetNoDelay)]: false,
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: null,
[Symbol(kBuffer)]: null,
[Symbol(kBufferCb)]: null,
[Symbol(kBufferGen)]: null,
[Symbol(kCapture)]: false,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0
},
domainThrown: false
}
2022-12-13T23:48:55: PM2 log: ===============================================================================
2022-12-13T23:48:55: PM2 log: — New PM2 Daemon started ----------------------------------------------------
2022-12-13T23:48:55: PM2 log: Time : Tue Dec 13 2022 23:48:55 GMT+0000 (Koordinierte Weltzeit)
2022-12-13T23:48:55: PM2 log: PM2 version : 5.2.2
2022-12-13T23:48:55: PM2 log: Node.js version : 14.21.1
2022-12-13T23:48:55: PM2 log: Current arch : x64
2022-12-13T23:48:55: PM2 log: PM2 home : /opt/pm2
2022-12-13T23:48:55: PM2 log: PM2 PID file : /opt/pm2/pm2.pid
2022-12-13T23:48:55: PM2 log: RPC socket file : /opt/pm2/rpc.sock
2022-12-13T23:48:55: PM2 log: BUS socket file : /opt/pm2/pub.sock
2022-12-13T23:48:55: PM2 log: Application log path : /opt/pm2/logs
2022-12-13T23:48:55: PM2 log: Worker Interval : 30000
2022-12-13T23:48:55: PM2 log: Process dump file : /opt/pm2/dump.pm2
2022-12-13T23:48:55: PM2 log: Concurrent actions : 2
2022-12-13T23:48:55: PM2 log: SIGTERM timeout : 1600
2022-12-13T23:48:55: PM2 log: ===============================================================================
2022-12-13T23:51:44: PM2 log: App [staging.example.com:0] starting in -cluster mode-
2022-12-13T23:51:44: PM2 log: App [staging.example.com:0] online
2022-12-13T23:51:44: PM2 log: App [staging.example.com:1] starting in -cluster mode-
2022-12-13T23:51:44: PM2 log: App [staging.example.com:1] online
2022-12-13T23:51:44: PM2 log: App [staging.example.com:2] starting in -cluster mode-
2022-12-13T23:51:45: PM2 log: App [staging.example.com:2] online
2022-12-13T23:51:45: PM2 log: App [staging.example.com:3] starting in -cluster mode-
2022-12-13T23:51:45: PM2 log: App [staging.example.com:3] online
It seemed to me that next.js got stuck on the “Activate New Deployment” Hook and strapi on the “Link Media Uploads + Activate New Deployment” hook.
It is so confusing, because it was running fine. And I do not quite understand which effect they have on each other.
What do you think? Where can I start?