Skip to main content

Friendica Support reshared this.


opensocial.at database data seems corrupt


!Friendica Support

It seems like I do have a problem with my opensocial.at database :-( .

The database ran full about a year ago (what a shame, I know :( ...) and I had to recover some of the data (otherwise I had to import a backup about ~12hours ago, I tried to avoid to loose posts for this period of time for my users ...). So I stopped it, started it with innodb_force_recovery=3, repaired it and checked it with mysqlcheck.

Two days ago, I updated the mariadb from 1.10.3 to 1.10.10 and now the problems occur again.

I stopped the MariaDB 1.10.3 container, updated it to 1.10.10 and started it again .. And MariaDB started a "crash recovery". I really don't know why... The crash recovery wasn't successfully (I tried it ~4 times), so I had to add innodb_force_recovery=3 again.

Now the database went up, but everytime I stopped it and started it again, the crash recovery appears again.

So after the instance was up and running, I dumped the whole database with mysqldump into one single *.sql , started a brand new MariaDB 1.10.10 and imported the dump again.

So far so good ..

But... unfortunately, after a restart, the crash recovery appears again. So I'm totally lost, what's now happening..

I noticed during the import that one batch of rows took about 2 hours to complete with the result query affected 0 rows.

here's my customized.cnf, which I'm using (it's a Hetzner root server with 64 GB RAM and 12 CPUs):
[mysqld]
query_cache_size = 0
query_cache_type = 0
performance_schema = ON
join_buffer_size = 140M
innodb_buffer_pool_size = 12G
innodb_log_buffer_size = 31M
innodb_log_file_size = 3G
table_open_cache = 1000
max_connections = 400
wait_timeout = 200
interactive_timeout = 4000

log-bin = mysqld-bin
transaction-isolation = READ-COMMITTED
binlog-format = ROW
skip-innodb-read-only-compressed = ON
innodb_read_only_compressed = OFF
innodb_use_native_aio = OFF

innodb_fast_shutdown=0
innodb_max_dirty_pages_pct=0
innodb_buffer_pool_dump_at_shutdown=1
innodb_buffer_pool_load_at_startup=1


For the new MariaDB instance, I didn't add any customization to avoid any wrong options.


currently, it makes a recovery again:
db_1     | 2022-12-16 10:41:00 0 [Note] InnoDB: Rolled back recovered transaction 371172                                                                                                                                                                                                    
db_1     | 2022-12-16 10:41:11 0 [Note] InnoDB: To roll back: 1 transactions, 7611737 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:26 0 [Note] InnoDB: To roll back: 1 transactions, 7608132 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:41 0 [Note] InnoDB: To roll back: 1 transactions, 7603940 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:46 0 [Note] DDL_LOG: Crash recovery executed 1 entries                                                                                                                                                                                                          
db_1     | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '0.0.0.0'.                                                                                                                                                                                                             
db_1     | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '::'.                                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:47 0 [Note] mariadbd: ready for connections.                                                                                                                                                                                                                    
db_1     | Version: '10.10.2-MariaDB-1:10.10.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution                                                                                                                                               
db_1     | 2022-12-16 10:41:56 0 [Note] InnoDB: To roll back: 1 transactions, 7601453 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:11 0 [Note] InnoDB: To roll back: 1 transactions, 7600146 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:26 0 [Note] InnoDB: To roll back: 1 transactions, 7598395 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:41 0 [Note] InnoDB: To roll back: 1 transactions, 7595967 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:56 0 [Note] InnoDB: To roll back: 1 transactions, 7594000 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:11 0 [Note] InnoDB: To roll back: 1 transactions, 7590646 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:26 0 [Note] InnoDB: To roll back: 1 transactions, 7585618 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:41 0 [Note] InnoDB: To roll back: 1 transactions, 7579929 rows
db_1     | 2022-12-16 10:43:56 0 [Note] InnoDB: To roll back: 1 transactions, 7574058 rows
db_1     | 2022-12-16 10:44:11 0 [Note] InnoDB: To roll back: 1 transactions, 7568214 rows

Friendica Support reshared this.

Wow, now it's super fast... thanks for the work. Do you have a way to receive donations?

Friendica Support reshared this.

@Torsten Torsten
Me?
No donations needed. I’m just happy to help :)

Friendica Admins reshared this.


Large database


!Friendica Admins
Hi everyone,

last week, I struggled with the size of my database for opensocial.at .. It went full (80GB) .. I added additional 10GB to the volume, but it seems to constantly increase and I think in a few weeks, I will be at the same situation like the last week.

Is there a possibility to shrink the database, or to wipe old data?
I recently deleted about 1.200 spam accounts (and set the register option to approval first ... lessons learned...). Are there leftovers inside the db which I can delete?

It's not that I don't have enough space, but I'm afraid of my backup borg-backup space in the near future ^^

Friendica Admins reshared this.

nope, there're no active relays "Das System hat derzeit keinerlei Relays abonniert."