Manual Docker Image Build - Containers exiting with error. #27061

akshayjain3450 · 2024-02-09T07:22:39Z

akshayjain3450
Feb 9, 2024

System Information:
Macbook Pro 2019 Intel i9
MacOS Sonoma
Docker Desktop

I checked out to tags/3.1.0 branch for the last stable release code of superset. Using this branch I am building a Docker image for myself. I have not made any changes to the code yet.

My docker build command is: docker build -t org/superset:0.0.1-exp.01 -f ./Dockerfile .

The build is successful. Now I am trying to use this image which I have built with compose YAML in the project. I have made only one change to the compose file: x-superset-image: &superset-image org/superset:0.0.1-exp.01 in line 17.

When I run the command docker compose up, a total of 10 containers are created.
My 4 containers exit and only 6 seem to be running.

Errors I get for the exited containers:
1. superset_init:

2024-02-09 12:45:12 Traceback (most recent call last):
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
2024-02-09 12:45:12     self.dialect.do_execute(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
2024-02-09 12:45:12 Loaded your LOCAL configuration at [/app/docker/pythonpath_dev/superset_config.py]
2024-02-09 12:45:12 
Cleaning up slice uuid from dashboard position json.. Done.      n json..
2024-02-09 12:45:12 
2024-02-09 12:45:12 Updated 0 pie chart labels.
2024-02-09 12:45:12 0 slices altered
2024-02-09 12:45:12 
Cleaning up slice uuid from dashboard position json.. Done.      n json..
2024-02-09 12:45:12 
2024-02-09 12:45:12 Updated 0 native filter configurations.
2024-02-09 12:45:12 Updated 0 filter sets with 0 filters.
2024-02-09 12:45:12 Upgraded 0 filters and 0 filter sets.
2024-02-09 12:45:12 Upgraded 0 slices.
2024-02-09 12:45:12 slices updated with no time_range_endpoints: 0
2024-02-09 12:45:12 >> Assign new UUIDs to tables...
2024-02-09 12:45:12 >> Drop intermediate columns...
2024-02-09 12:45:12     cursor.execute(statement, parameters)
2024-02-09 12:45:12 psycopg2.errors.DeadlockDetected: deadlock detected
2024-02-09 12:45:12 DETAIL:  Process 78 waits for AccessExclusiveLock on relation 16405 of database 16384; blocked by process 82.
2024-02-09 12:45:12 Process 82 waits for AccessShareLock on relation 16413 of database 16384; blocked by process 78.
2024-02-09 12:45:12 HINT:  See server log for query details.
2024-02-09 12:45:12 
2024-02-09 12:45:12 
2024-02-09 12:45:12 The above exception was the direct cause of the following exception:
2024-02-09 12:45:12 
2024-02-09 12:45:12 Traceback (most recent call last):
2024-02-09 12:45:12   File "/usr/local/bin/superset", line 33, in <module>
2024-02-09 12:45:12     sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
2024-02-09 12:45:12     return self.main(*args, **kwargs)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
2024-02-09 12:45:12     rv = self.invoke(ctx)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
2024-02-09 12:45:12     return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
2024-02-09 12:45:12     return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
2024-02-09 12:45:12     return ctx.invoke(self.callback, **ctx.params)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
2024-02-09 12:45:12     return __callback(*args, **kwargs)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
2024-02-09 12:45:12     return f(get_current_context(), *args, **kwargs)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/flask/cli.py", line 357, in decorator
2024-02-09 12:45:12     return __ctx.invoke(f, *args, **kwargs)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
2024-02-09 12:45:12     return __callback(*args, **kwargs)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/flask_migrate/cli.py", line 149, in upgrade
2024-02-09 12:45:12     _upgrade(directory, revision, sql, tag, x_arg)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/flask_migrate/__init__.py", line 98, in wrapped
2024-02-09 12:45:12     f(*args, **kwargs)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/flask_migrate/__init__.py", line 185, in upgrade
2024-02-09 12:45:12     command.upgrade(config, revision, sql=sql, tag=tag)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/command.py", line 294, in upgrade
2024-02-09 12:45:12     script.run_env()
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/script/base.py", line 490, in run_env
2024-02-09 12:45:12     util.load_python_file(self.dir, "env.py")
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
2024-02-09 12:45:12     module = load_module_py(module_id, path)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/util/compat.py", line 184, in load_module_py
2024-02-09 12:45:12     spec.loader.exec_module(module)
2024-02-09 12:45:12   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
2024-02-09 12:45:12   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
2024-02-09 12:45:12   File "/app/superset/extensions/../migrations/env.py", line 126, in <module>
2024-02-09 12:45:12     run_migrations_online()
2024-02-09 12:45:12   File "/app/superset/extensions/../migrations/env.py", line 118, in run_migrations_online
2024-02-09 12:45:12     context.run_migrations()
2024-02-09 12:45:12   File "<string>", line 8, in run_migrations
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
2024-02-09 12:45:12     self.get_context().run_migrations(**kw)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
2024-02-09 12:45:12     step.migration_fn(**kw)
2024-02-09 12:45:12   File "/app/superset/migrations/versions/2023-09-15_12-58_4b85906e5b91_add_on_delete_cascade_for_dashboard_roles.py", line 50, in upgrade
2024-02-09 12:45:12     redefine(foreign_key, on_delete="CASCADE")
2024-02-09 12:45:12   File "/app/superset/migrations/shared/constraints.py", line 50, in redefine
2024-02-09 12:45:12     batch_op.create_foreign_key(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/contextlib.py", line 126, in __exit__
2024-02-09 12:45:12     next(self.gen)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/operations/base.py", line 336, in batch_alter_table
2024-02-09 12:45:12     impl.flush()
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/operations/batch.py", line 86, in flush
2024-02-09 12:45:12     fn(*arg, **kw)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/ddl/impl.py", line 266, in drop_constraint
2024-02-09 12:45:12     self._exec(schema.DropConstraint(const))
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/alembic/ddl/impl.py", line 146, in _exec
2024-02-09 12:45:12     return conn.execute(construct, multiparams)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1306, in execute
2024-02-09 12:45:12     return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
2024-02-09 12:45:12     return connection._execute_ddl(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1398, in _execute_ddl
2024-02-09 12:45:12     ret = self._execute_context(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
2024-02-09 12:45:12     self._handle_dbapi_exception(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
2024-02-09 12:45:12     util.raise_(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
2024-02-09 12:45:12     raise exception
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
2024-02-09 12:45:12     self.dialect.do_execute(
2024-02-09 12:45:12   File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
2024-02-09 12:45:12     cursor.execute(statement, parameters)
2024-02-09 12:45:12 sqlalchemy.exc.OperationalError: (psycopg2.errors.DeadlockDetected) deadlock detected
2024-02-09 12:45:12 DETAIL:  Process 78 waits for AccessExclusiveLock on relation 16405 of database 16384; blocked by process 82.
2024-02-09 12:45:12 Process 82 waits for AccessShareLock on relation 16413 of database 16384; blocked by process 78.
2024-02-09 12:45:12 HINT:  See server log for query details.
2024-02-09 12:45:12 
2024-02-09 12:45:12 [SQL: ALTER TABLE dashboard_roles DROP CONSTRAINT dashboard_roles_role_id_fkey]
2024-02-09 12:45:12 (Background on this error at: https://sqlalche.me/e/14/e3q8)

2. superset_node:

2024-02-09 12:46:32 found 0 vulnerabilities
2024-02-09 12:46:20 npm WARN config optional Use `--omit=optional` to exclude optional dependencies, or
2024-02-09 12:46:20 npm WARN config `--include=optional` to include them.
2024-02-09 12:46:20 npm WARN config 
2024-02-09 12:46:20 npm WARN config     Default value does install optional deps unless otherwise omitted.
2024-02-09 12:46:20 npm WARN using --force Recommended protections disabled.
2024-02-09 12:46:32 npm notice 
2024-02-09 12:46:32 npm notice New major version of npm available! 8.19.4 -> 10.4.0
2024-02-09 12:46:32 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.4.0>
2024-02-09 12:46:32 npm notice Run `npm install -g [email protected]` to update!
2024-02-09 12:46:32 npm notice 
2024-02-09 12:46:32 npm WARN config optional Use `--omit=optional` to exclude optional dependencies, or
2024-02-09 12:46:32 npm WARN config `--include=optional` to include them.
2024-02-09 12:46:32 npm WARN config 
2024-02-09 12:46:32 npm WARN config     Default value does install optional deps unless otherwise omitted.
2024-02-09 12:46:32 npm WARN using --force Recommended protections disabled.
2024-02-09 12:47:06 npm ERR! code ENOTEMPTY
2024-02-09 12:47:06 npm ERR! syscall rename
2024-02-09 12:47:06 npm ERR! path /app/superset-frontend/packages/superset-ui-core/node_modules/whatwg-fetch
2024-02-09 12:47:06 npm ERR! dest /app/superset-frontend/packages/superset-ui-core/node_modules/.whatwg-fetch-IjyaYMEM
2024-02-09 12:47:06 npm ERR! errno -39
2024-02-09 12:47:06 npm ERR! ENOTEMPTY: directory not empty, rename '/app/superset-frontend/packages/superset-ui-core/node_modules/whatwg-fetch' -> '/app/superset-frontend/packages/superset-ui-core/node_modules/.whatwg-fetch-IjyaYMEM'
2024-02-09 12:47:06 
2024-02-09 12:47:06 npm ERR! A complete log of this run can be found in:
2024-02-09 12:47:06 npm ERR!     /root/.npm/_logs/2024-02-09T07_16_32_945Z-debug-0.log

3. superset_tests_worker:

2024-02-09 12:45:02   warnings.warn(
2024-02-09 12:45:02 2024-02-09 07:15:02,475:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: Connection refused
2024-02-09 12:45:02     Is the server running on that host and accepting TCP/IP connections?
2024-02-09 12:45:02 connection to server at "localhost" (127.0.0.1), port 5432 failed: FATAL:  database "test" does not exist

4. superset_websocket:


2024-02-09 12:44:49 {"date":"Fri Feb 09 2024 07:14:49 GMT+0000 (Coordinated Universal Time)","error":{},"exception":true,"level":"error","message":"uncaughtException: Please provide a JWT secret at least 32 bytes long\nError: Please provide a JWT secret at least 32 bytes long\n    at Object.<anonymous> (/home/superset-websocket/dist/index.js:76:11)\n    at Module._compile (node:internal/modules/cjs/loader:1198:14)\n    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)\n    at Module.load (node:internal/modules/cjs/loader:1076:32)\n    at Function.Module._load (node:internal/modules/cjs/loader:911:12)\n    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n    at node:internal/main/run_main_module:22:47","os":{"loadavg":[1.02,0.22,0.16],"uptime":17174.8},"process":{"argv":["/usr/local/bin/node","/home/superset-websocket/dist/index.js","start"],"cwd":"/home/superset-websocket","execPath":"/usr/local/bin/node","gid":1000,"memoryUsage":{"arrayBuffers":74963,"external":1802731,"heapTotal":29921280,"heapUsed":15723776,"rss":59682816},"pid":18,"uid":1000,"version":"v16.20.2"},"stack":"Error: Please provide a JWT secret at least 32 bytes long\n    at Object.<anonymous> (/home/superset-websocket/dist/index.js:76:11)\n    at Module._compile (node:internal/modules/cjs/loader:1198:14)\n    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)\n    at Module.load (node:internal/modules/cjs/loader:1076:32)\n    at Function.Module._load (node:internal/modules/cjs/loader:911:12)\n    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n    at node:internal/main/run_main_module:22:47","trace":[{"column":11,"file":"/home/superset-websocket/dist/index.js","function":null,"line":76,"method":null,"native":false},{"column":14,"file":"node:internal/modules/cjs/loader","function":"Module._compile","line":1198,"method":"_compile","native":false},{"column":10,"file":"node:internal/modules/cjs/loader","function":"Module._extensions..js","line":1252,"method":".js","native":false},{"column":32,"file":"node:internal/modules/cjs/loader","function":"Module.load","line":1076,"method":"load","native":false},{"column":12,"file":"node:internal/modules/cjs/loader","function":"Module._load","line":911,"method":"_load","native":false},{"column":12,"file":"node:internal/modules/run_main","function":"Function.executeUserEntryPoint [as runMain]","line":81,"method":"executeUserEntryPoint [as runMain]","native":false},{"column":47,"file":"node:internal/main/run_main_module","function":null,"line":22,"method":null,"native":false}]}
2024-02-09 12:44:49 config.json file not found

Looking for support on this.

Answered by mistercrunch

Feb 12, 2024

Database deadlocks, looks like you have a migration (touching dashboard_roles) conflicting with the app itself. Stop the app, kill all sessions in the database, get a clean run on the migrations, and restart the app.

View full answer

akshayjain3450 · 2024-02-09T07:33:51Z

akshayjain3450
Feb 9, 2024
Author

@michael-s-molina @rusackas @betodealmeida @john-bodley @geido @mistercrunch
Need your help with this.

0 replies

mistercrunch · 2024-02-12T16:59:01Z

mistercrunch
Feb 12, 2024
Collaborator

Database deadlocks, looks like you have a migration (touching dashboard_roles) conflicting with the app itself. Stop the app, kill all sessions in the database, get a clean run on the migrations, and restart the app.

6 replies

akshayjain3450 Feb 13, 2024
Author

I have resolved all the errors I have faced. I have realized the documentation is not good enough for developers to build their custom images of superset and try them on their machines. I would like to open an issue on this and contribute towards providing the solution to that. @rusackas

akshayjain3450 Feb 13, 2024
Author

@mistercrunch Your solutions seem fine, but I am facing the same issue again and again even when I properly close down all the containers and remove the volumes. Is there any permanent solution to this that I can use in my development?
Your solution seems more oriented towards doing individual tasks and not running the application using docker compose. So can you please tell me why is this happening with docker compose.

rusackas Feb 13, 2024
Collaborator

Contributions welcome! Thanks for offering to get involved @akshayjain3450 :)

geido Feb 13, 2024
Collaborator

+1!

AnneSophieTreton Sep 8, 2024

@akshayjain3450 I am facing the same error with superset_init, what was your solution please ? Thank you :)

mistercrunch · 2024-09-09T16:20:56Z

mistercrunch
Sep 9, 2024
Collaborator

what was your solution please ?

The solution is spelled out pretty clearly. Kill the app, and all database sessions (restarting the database will kill all sessions) and then fire up superset_init

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Manual Docker Image Build - Containers exiting with error. #27061

{{title}}

Replies: 3 comments 6 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Manual Docker Image Build - Containers exiting with error. #27061

akshayjain3450 Feb 9, 2024

Replies: 3 comments · 6 replies

akshayjain3450 Feb 9, 2024 Author

mistercrunch Feb 12, 2024 Collaborator

akshayjain3450 Feb 13, 2024 Author

akshayjain3450 Feb 13, 2024 Author

rusackas Feb 13, 2024 Collaborator

geido Feb 13, 2024 Collaborator

AnneSophieTreton Sep 8, 2024

mistercrunch Sep 9, 2024 Collaborator

akshayjain3450
Feb 9, 2024

Replies: 3 comments 6 replies

akshayjain3450
Feb 9, 2024
Author

mistercrunch
Feb 12, 2024
Collaborator

akshayjain3450 Feb 13, 2024
Author

akshayjain3450 Feb 13, 2024
Author

rusackas Feb 13, 2024
Collaborator

geido Feb 13, 2024
Collaborator

mistercrunch
Sep 9, 2024
Collaborator