Friendica Admins reshared this.
Slow photo requests / inbox requests
Hi !Friendica Admins ,
I'm currently tracing down performance issues, I' currently suffering from on my nodes opensocial.at and friendica.me .
It seems like the main performance problems are
- delivering photos
- [code]/inbox/code] requests
You can see some peeks and some long runners here:
The long runners (10sec) are the /inbox requests, the peaks are delivering photos after opening conversations, network pane, ...
Do you have any hints how I could improve these two types of requests?
The database is currently using ~50GB RAM, I set them as high as possible.
Friendica Developers reshared this.
Update JSON LD
!Friendica Developers
@Hypolite Petovan already fixed a current JSON-LD issue with https://git.friendi.ca/friendica/php-json-ld/pulls/1
Do we need a 1.1.2 release to upgrade it in the Friendica composer.lock file as well?
The logs of my instances are flooded with this error, that's why I'm asking *g*
@Hypolite Petovan already fixed a current JSON-LD issue with https://git.friendi.ca/friendica/php-json-ld/pulls/1
Do we need a 1.1.2 release to upgrade it in the Friendica composer.lock file as well?
The logs of my instances are flooded with this error, that's why I'm asking *g*
Add support for local files for unsecured document loading
- $http_response_header isn't set in the local file read case Address https://github.com/friendica/friendica/issues/12728#issuecomment-1407623586Friendica
like this
Friendica Developers reshared this.
Friendica Developers reshared this.
Friendica Developers reshared this.
BaseURL components (hostname, SSL policy, urlpath)
!Friendica Developers
I'm currently reducing the whole
Do we really need the ssl_policy, urlpath and hostname separate from the
I will use for the
The only thing, which I'm unsure is the
So I would drop it as well and merge all config entries into the
Additionally, I will replace the Exception with a "CRITICAL" log entry to avoid a WSOD.
I'm currently reducing the whole
BaseUrl.php code massively.Do we really need the ssl_policy, urlpath and hostname separate from the
system.url? I will use for the
BaseUrl.php, based on the system.url a "real" UriInterface as $this->url, so we don't need saving the scheme, urlpath and hostname separately anymore. They are just useful for the install process but must not be changed afterwards.The only thing, which I'm unsure is the
ssl_policy, because if someone changes it afterwards in the admin site, all URL in all contacts and photos will get updated. But the question is => is this even allowed? I think this could brick the access over federation because the base-url of each entry isn't right anymore. And it isn't supported when the policy is changed by console.So I would drop it as well and merge all config entries into the
system.url.Additionally, I will replace the Exception with a "CRITICAL" log entry to avoid a WSOD.
Friendica Developers reshared this.
@Philipp Holzer For a local installation, like
https://friendica.local you definitely need to ignore SSL/certificate errors because they are mostly only self-signed.Friendica Developers reshared this.
I know and I'm using local Vagrant with 
https and there is no check at all in the code that would fail here. It works fine Roland Häder likes this.
Opensocial.at reshared this.
Friendica.me | Opensocial.at Short Downtime because of DB adjustements
!friendica.me , !Opensocial.at
I do have to restart the Database instances because they are too resource hungry.
There's a short downtime possible.
I do have to restart the Database instances because they are too resource hungry.
There's a short downtime possible.
friendica.me reshared this.
Friendica Support reshared this.
opensocial.at database data seems corrupt
!Friendica Support
It seems like I do have a problem with my opensocial.at database
.
The database ran full about a year ago (what a shame, I know :( ...) and I had to recover some of the data (otherwise I had to import a backup about ~12hours ago, I tried to avoid to loose posts for this period of time for my users ...). So I stopped it, started it with
Two days ago, I updated the mariadb from 1.10.3 to 1.10.10 and now the problems occur again.
I stopped the MariaDB 1.10.3 container, updated it to 1.10.10 and started it again .. And MariaDB started a "crash recovery". I really don't know why... The crash recovery wasn't successfully (I tried it ~4 times), so I had to add
Now the database went up, but everytime I stopped it and started it again, the crash recovery appears again.
So after the instance was up and running, I dumped the whole database with
So far so good ..
But... unfortunately, after a restart, the crash recovery appears again. So I'm totally lost, what's now happening..
I noticed during the import that one batch of rows took about 2 hours to complete with the result query affected 0 rows.
here's my
For the new MariaDB instance, I didn't add any customization to avoid any wrong options.
currently, it makes a recovery again:
It seems like I do have a problem with my opensocial.at database
.The database ran full about a year ago (what a shame, I know :( ...) and I had to recover some of the data (otherwise I had to import a backup about ~12hours ago, I tried to avoid to loose posts for this period of time for my users ...). So I stopped it, started it with
innodb_force_recovery=3, repaired it and checked it with mysqlcheck.Two days ago, I updated the mariadb from 1.10.3 to 1.10.10 and now the problems occur again.
I stopped the MariaDB 1.10.3 container, updated it to 1.10.10 and started it again .. And MariaDB started a "crash recovery". I really don't know why... The crash recovery wasn't successfully (I tried it ~4 times), so I had to add
innodb_force_recovery=3 again.Now the database went up, but everytime I stopped it and started it again, the crash recovery appears again.
So after the instance was up and running, I dumped the whole database with
mysqldump into one single *.sql , started a brand new MariaDB 1.10.10 and imported the dump again.So far so good ..
But... unfortunately, after a restart, the crash recovery appears again. So I'm totally lost, what's now happening..
I noticed during the import that one batch of rows took about 2 hours to complete with the result query affected 0 rows.
here's my
customized.cnf, which I'm using (it's a Hetzner root server with 64 GB RAM and 12 CPUs):[mysqld]
query_cache_size = 0
query_cache_type = 0
performance_schema = ON
join_buffer_size = 140M
innodb_buffer_pool_size = 12G
innodb_log_buffer_size = 31M
innodb_log_file_size = 3G
table_open_cache = 1000
max_connections = 400
wait_timeout = 200
interactive_timeout = 4000
log-bin = mysqld-bin
transaction-isolation = READ-COMMITTED
binlog-format = ROW
skip-innodb-read-only-compressed = ON
innodb_read_only_compressed = OFF
innodb_use_native_aio = OFF
innodb_fast_shutdown=0
innodb_max_dirty_pages_pct=0
innodb_buffer_pool_dump_at_shutdown=1
innodb_buffer_pool_load_at_startup=1For the new MariaDB instance, I didn't add any customization to avoid any wrong options.
currently, it makes a recovery again:
db_1 | 2022-12-16 10:41:00 0 [Note] InnoDB: Rolled back recovered transaction 371172
db_1 | 2022-12-16 10:41:11 0 [Note] InnoDB: To roll back: 1 transactions, 7611737 rows
db_1 | 2022-12-16 10:41:26 0 [Note] InnoDB: To roll back: 1 transactions, 7608132 rows
db_1 | 2022-12-16 10:41:41 0 [Note] InnoDB: To roll back: 1 transactions, 7603940 rows
db_1 | 2022-12-16 10:41:46 0 [Note] DDL_LOG: Crash recovery executed 1 entries
db_1 | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '0.0.0.0'.
db_1 | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '::'.
db_1 | 2022-12-16 10:41:47 0 [Note] mariadbd: ready for connections.
db_1 | Version: '10.10.2-MariaDB-1:10.10.2+maria~ubu2204' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
db_1 | 2022-12-16 10:41:56 0 [Note] InnoDB: To roll back: 1 transactions, 7601453 rows
db_1 | 2022-12-16 10:42:11 0 [Note] InnoDB: To roll back: 1 transactions, 7600146 rows
db_1 | 2022-12-16 10:42:26 0 [Note] InnoDB: To roll back: 1 transactions, 7598395 rows
db_1 | 2022-12-16 10:42:41 0 [Note] InnoDB: To roll back: 1 transactions, 7595967 rows
db_1 | 2022-12-16 10:42:56 0 [Note] InnoDB: To roll back: 1 transactions, 7594000 rows
db_1 | 2022-12-16 10:43:11 0 [Note] InnoDB: To roll back: 1 transactions, 7590646 rows
db_1 | 2022-12-16 10:43:26 0 [Note] InnoDB: To roll back: 1 transactions, 7585618 rows
db_1 | 2022-12-16 10:43:41 0 [Note] InnoDB: To roll back: 1 transactions, 7579929 rows
db_1 | 2022-12-16 10:43:56 0 [Note] InnoDB: To roll back: 1 transactions, 7574058 rows
db_1 | 2022-12-16 10:44:11 0 [Note] InnoDB: To roll back: 1 transactions, 7568214 rowslike this
Friendica Support reshared this.
Wow, now it's super fast... thanks for the work. Do you have a way to receive donations?
Raroun likes this.
Friendica Support reshared this.
Friendica Developers reshared this.
Questions about /display/
!Friendica Developers
@Michael Vogel - I'm currently moving the
As far as I can see, there are two main-routes for this controller:
and as far as I can say, it would be better if the second route should have an own root, like `dfrn_item/{uri-id}[/conversation.atom`, as it's a DFRN only logic.
Am I right so far?
If so, can I move it to this new path or is this path necessary for the federation usage, means it could be called from other servers too?
I just found calls with prefix
@Michael Vogel - I'm currently moving the
mod/display.phpAs far as I can see, there are two main-routes for this controller:
-
/display/{guid} -
/display/feed-item/{uri-id}.atom[/conversation.atom]
and as far as I can say, it would be better if the second route should have an own root, like `dfrn_item/{uri-id}[/conversation.atom`, as it's a DFRN only logic.
Am I right so far?
If so, can I move it to this new path or is this path necessary for the federation usage, means it could be called from other servers too?
I just found calls with prefix
DI::baseUrl(), so I guess it's "just" for the server itself, but I'm not sure ...
Friendica Developers reshared this.
@Michael Vogel / @Hypolite Petovan
I still struggle at
It's the only place, where the
But I don't find any place inside Friendica, where we call
Do you know if this is till in use and if so, what's the meaning of it?
If we wouldn't use it anymore, there's a lot of code at
I still struggle at
mod/update_display.phpIt's the only place, where the
display_content from mod/display.php is called with the parameter $update = true.But I don't find any place inside Friendica, where we call
/update_display?p=<profile_uid>.Do you know if this is till in use and if so, what's the meaning of it?
If we wouldn't use it anymore, there's a lot of code at
mod/display.php, which I could delete in that case..This entry was edited (6 months ago)
Friendica Developers reshared this.
Got it, it was a little bit hidden *gg*:
https://github.com/friendica/friendica/blob/4a626ef10176a6f4fca0367cd890f836c2ade44a/src/Content/Conversation.php#L498-L505
and then
https://github.com/friendica/friendica/blob/b002f6f027cce0d9901c5129470e5cf41bb334c4/view/js/main.js#L595
https://github.com/friendica/friendica/blob/4a626ef10176a6f4fca0367cd890f836c2ade44a/src/Content/Conversation.php#L498-L505
and then
https://github.com/friendica/friendica/blob/b002f6f027cce0d9901c5129470e5cf41bb334c4/view/js/main.js#L595
friendica/Conversation.php at 4a626ef10176a6f4fca0367cd890f836c2ade44a ยท friendica/friendica
Friendica Communications Platform. Contribute to friendica/friendica development by creating an account on GitHub.GitHub
This entry was edited (6 months ago)
Hypolite Petovan likes this.
Friendica Developers reshared this.
API usage for Friendica Frontend
!Friendica Developers
And another question
Currently, we strictly separate between API calls and the Frontend.
But, Why?
I'm currently moving the events to the Module directory.
And there's already an API for it, but as far as I understand @Michael Vogel , we must not mix API and Frontend.
But as far as I know, in modern environments, there's a backend part (= API), and a frontend part (= view)
The own frontend still uses the same API. So we don't have to implement the same business logic at different places. As I said, I came across at Event, where this issue produces significant double-code.
So at least what I would like (in a final state:
I know, this would be a BIG change
And another question
Currently, we strictly separate between API calls and the Frontend.
But, Why?
I'm currently moving the events to the Module directory.
And there's already an API for it, but as far as I understand @Michael Vogel , we must not mix API and Frontend.
But as far as I know, in modern environments, there's a backend part (= API), and a frontend part (= view)
The own frontend still uses the same API. So we don't have to implement the same business logic at different places. As I said, I came across at Event, where this issue produces significant double-code.
So at least what I would like (in a final state
:- API ... Create, Update, Delete, Get, GetMultiple for our Data model, or "everything without HTML output"
- Module ... Everything, which produces HTML output
I know, this would be a BIG change
like this
Friendica Developers reshared this.
In modern environments the frontend is calling the API. We are not there yet.
I would like having that separation since this would allow us to just test all API calls to make sure that the frontend is working as expected.
I would like having that separation since this would allow us to just test all API calls to make sure that the frontend is working as expected.
Grischa likes this.
Friendica Developers reshared this.
@Michael Vogel
What is the showstopper for that? Cause that is also a thing for a new Webdesign. So The Web sould be separate from the code with all the magic.
@Philipp Holzer
What is the showstopper for that? Cause that is also a thing for a new Webdesign. So The Web sould be separate from the code with all the magic.
@Philipp Holzer
Friendica Developers reshared this.
API is a misnomer in our case because it can designate several things:
- Our internal API (what you described in your post)
- The Twitter API endpoints we support
- The Mastodon API endpoints we support
- The GNU Social API endpoints we support
- The Friendica-specific API endpoints we expose (mainly for Friendiqa)
We definitely lack a unified internal API but thatโs supposed to be taken care of by the Repository/Factory/Model/Entity structure that we should keep expanding.
- Our internal API (what you described in your post)
- The Twitter API endpoints we support
- The Mastodon API endpoints we support
- The GNU Social API endpoints we support
- The Friendica-specific API endpoints we expose (mainly for Friendiqa)
We definitely lack a unified internal API but thatโs supposed to be taken care of by the Repository/Factory/Model/Entity structure that we should keep expanding.
This entry was edited (7 months ago)
Friendica Developers reshared this.
@Hypolite Petovan
why don't separate the api for Twitter, Mastodon, Gnu etc. and of course an api to communicate with the frontend.
No mixing the api together. Activateing the maybe Mastodon APi as main class with subclasses in one folder and subfolders.
@Philipp Holzer @Michael Vogel
why don't separate the api for Twitter, Mastodon, Gnu etc. and of course an api to communicate with the frontend.
No mixing the api together. Activateing the maybe Mastodon APi as main class with subclasses in one folder and subfolders.
@Philipp Holzer @Michael Vogel
This entry was edited (7 months ago)
Friendica Developers reshared this.
So far weโve done a good job at having the business code identical for the Twitter and Mastodon API endpoints. There are some legacy Friendica endpoints (mostly the asynchronous endpoints used by Javascript) that are redundant with the page modules.
Friendica Developers reshared this.
@Hypolite Petovan
Can this endpoint not be closed and moved to an other area. So that you have the main friendica core who will do the internal magic. On the other hand the API section who is communicating with the core. Additional an internal API who is communicating with the core and the frontend section. On all APIs there is only data communication.
The Frontend Section is an additional area who will communicate with the frontend API and the user.
So the magic on the frontend section will be created only in the frontend and the API communication will only send data ... No html, no Jscript and no other magic.
In that case all three sections (Core, API, Frontend) could be developed separate with less interference between them.
Sure if you need a new function... First core the API and if needed then frontend
@Philipp Holzer @Michael Vogel
Can this endpoint not be closed and moved to an other area. So that you have the main friendica core who will do the internal magic. On the other hand the API section who is communicating with the core. Additional an internal API who is communicating with the core and the frontend section. On all APIs there is only data communication.
The Frontend Section is an additional area who will communicate with the frontend API and the user.
So the magic on the frontend section will be created only in the frontend and the API communication will only send data ... No html, no Jscript and no other magic.
In that case all three sections (Core, API, Frontend) could be developed separate with less interference between them.
Sure if you need a new function... First core the API and if needed then frontend
@Philipp Holzer @Michael Vogel
Friendica Developers reshared this.
I don't think we need to go that far. Separating the web frontend from the API would be a massive undertaking at this point, and also there's a trend where even Javascript frontends are starting to request already formed HTML from the backend because it turns out it's faster this way. Which means we're both behind and ahead of the curve!
However we do need to extract the business logic in a unique place so that we can reference it from the places it is duplicated.
However we do need to extract the business logic in a unique place so that we can reference it from the places it is duplicated.
Friendica Developers reshared this.
@Hypolite Petovan
I think this is very necessary.
Passing preformatted web content makes you very inflexible since you have to go directly to the core. That's where I see the problem, that you either have a good coder for the core but not a good designer. The other way round a designer of the frontend can't write a good core code.
For this reason the Smarty framework is currently running in Friendica, which as I understand the framework wants to separate this.
If you want to speed up the whole thing, you have to get away from PHP code which is executed at runtime and create static pages. Peppered with lots of javascript which offloads/executes the execution to the client.
But ok that's my opinion on the subject.
@Philipp Holzer @Michael Vogel
I think this is very necessary.
Passing preformatted web content makes you very inflexible since you have to go directly to the core. That's where I see the problem, that you either have a good coder for the core but not a good designer. The other way round a designer of the frontend can't write a good core code.
For this reason the Smarty framework is currently running in Friendica, which as I understand the framework wants to separate this.
If you want to speed up the whole thing, you have to get away from PHP code which is executed at runtime and create static pages. Peppered with lots of javascript which offloads/executes the execution to the client.
But ok that's my opinion on the subject.
@Philipp Holzer @Michael Vogel
Roland Häder likes this.
Roland Häder reshared this.
Friendica Developers reshared this.
Thankfully, we can have both HTML modules and API endpoints! As long as we don't duplicate business code, which is what Philipp has been encountering.
On the other hand, writing a full Javascript web frontend is a massive endeavor, there currently are almost 500 web frontend routes that each represent a part of a Friendica feature. They all would need to have a Javascript counterpart, without even counting the corresponding bespoke API endpoints that would need to be created.
On the other hand, writing a full Javascript web frontend is a massive endeavor, there currently are almost 500 web frontend routes that each represent a part of a Friendica feature. They all would need to have a Javascript counterpart, without even counting the corresponding bespoke API endpoints that would need to be created.
Friendica Developers reshared this.
Add-on refactoring
!Friendica Developers
So this are my first thoughs about the Addons:
This is it so far..
I would like to start with creating the events.config.php and replace all text with constants.. I think this is a non invasive behaviour, but gives some insights about the next steps
So this are my first thoughs about the Addons:
- Metadata about each add-on should be in an extra file, I would prefer something like
addon.json - I would like to replace the whole
Hookclass with a proper Registration class, which is loaded when activating the Add-on. So we can remove the whole Hook table. - All Events in Friendica should be in an
events.config.phpwith a proper description (maybe as constants?) - for legacy purpose, I would create a wrapper class like the LegacyModule class, which would execute the corresponding php-functions
- I'm thinking about a giving each Event a specific interface (not knowing how tbh currently, maybe the corresponding interface setting as a value for the Event inside the events.config.php?), so each event and each add-on will know exactly what parameters and which result is necessary
This is it so far..
I would like to start with creating the events.config.php and replace all text with constants.. I think this is a non invasive behaviour, but gives some insights about the next steps
Hypolite Petovan likes this.
Friendica Developers reshared this.
Like always this is kind of foggy for me But I guess it will be a good idea to implement it in parallel to the current mechanism and then to move the addons step by step.
Du you have got an idea how to keep the folder structure in a way that all addons will stay in the addon folder?
But I guess it will be a good idea to implement it in parallel to the current mechanism and then to move the addons step by step.Du you have got an idea how to keep the folder structure in a way that all addons will stay in the addon folder?
Friendica Developers reshared this.
The folder structure wonโt change, but instead of a list of functions, addons will become classes.
Philipp Holzer likes this.
Friendica Developers reshared this.
Friendica codebase restructure
!Friendica Developers
Based on the PR https://github.com/friendica/friendica/pull/12075
@Hypolite Petovan is right, I should have started this conversation first, but it made fun to restructure our code, so it is what it is *gg*
What's the issue/my pain:
I'm thinking about introducing the PSR-7 Request/Response interfaces and create an Emitter, so there's no way out anymore directly inside the code, everything is done by the emitter. With this pattern, we could add all the authentication stuff inside a "Middleware" and get rid of the different auth-places (
So, now I tried to think where to start. We would need a
And I try to start to "really" encapsulate the domain-based directories inside
In the end, I think we should have some core directories at
I hope I didn't loose you at least, I just wrote down my path to this PR :)
PS: there're "just" ~200 code changes, the rest is because of the messages.po recreation don't worry :)
Based on the PR https://github.com/friendica/friendica/pull/12075
@Hypolite Petovan is right, I should have started this conversation first, but it made fun to restructure our code, so it is what it is *gg*
What's the issue/my pain:
I'm thinking about introducing the PSR-7 Request/Response interfaces and create an Emitter, so there's no way out anymore directly inside the code, everything is done by the emitter. With this pattern, we could add all the authentication stuff inside a "Middleware" and get rid of the different auth-places (
App, Security\Authentication, Module\BaseApi)So, now I tried to think where to start. We would need a
Middleware directory for all the auth/base-load things, but we already do have a mix of domain-based directories (TowFactorAuth), functionallity based directories (Model/Object/Factory/Collection/Capabilities) and and core directories (Core, App, Console, Worker)...And I try to start to "really" encapsulate the domain-based directories inside
Friendica\Library, so we get rid of some mixings, having all domain based classes inside one directory.In the end, I think we should have some core directories at
src, directing everyone to the right place.I hope I didn't loose you at least, I just wrote down my path to this PR :)
PS: there're "just" ~200 code changes, the rest is because of the messages.po recreation don't worry :)
Restructure Friendica Base by nupplaphil ยท Pull Request #12075 ยท friendica/friendica
I always struggle with our codebase about where to find what content .. Because we moved from pattern to pattern, we left a lot of leftovers in our codebase. This PR should restructure our codebase...GitHub
This entry was edited (7 months ago)
Friendica Developers reshared this.
So do you agree with the restructuring or shall I postpone it after mod/ is restructured? I think first, but I'm not sure 
Friendica Developers reshared this.
Friendica Developers reshared this.
Friendica Woodpecker / CI in GitHub bricked
!Friendica Developers
I'm sorry, I bricked the CI with the latest Woodpecker change.. Now instead that we don't create Archives at all nodes, we now don't execute some pipelines at all and they are "stuck forever" and blocking new executions.
I'm not sure if I can fix it today, so here's the announcement
I will kick the pipelines from time to time to make PR checks possible, but the "merge" pipelines aren't working (so no new archive appears currently..)
I'm sorry, I bricked the CI with the latest Woodpecker change.. Now instead that we don't create Archives at all nodes, we now don't execute some pipelines at all and they are "stuck forever" and blocking new executions.
I'm not sure if I can fix it today, so here's the announcement
I will kick the pipelines from time to time to make PR checks possible, but the "merge" pipelines aren't working (so no new archive appears currently..)
This entry was edited (7 months ago)
Hypolite Petovan likes this.
Friendica Developers reshared this.
I think I got it! :)
It was a mix of wrong environment variable setup for
--> as far as I can see, all Archives are now deployed at https://files.friendi.ca as expected :)
It was a mix of wrong environment variable setup for
$WOODPECKER_LABELS (added additional " for the value, which got escaped and added to the value) and using the `label` filter in Woodpecker, not knowing that this feature is only in the nightly builds yet--> as far as I can see, all Archives are now deployed at https://files.friendi.ca as expected :)
Hypolite Petovan likes this.
Friendica Developers reshared this.
Yet another Git-Repo :D
!Friendica Developers
@Tobias - I'd like to commit changes for the computed "docs" page at https://git.friendi.ca/friendica/docs.git
Can you create this repo for me and make me admin for it? Thx
@Tobias - I'd like to commit changes for the computed "docs" page at https://git.friendi.ca/friendica/docs.git
Can you create this repo for me and make me admin for it? Thx
Roland Häder likes this.
Friendica Developers reshared this.
docs.friendi.ca
!Friendica Developers
Hi, I'd like to add a new domain `docs.friendi.ca` where we store the whole documentation. It should be versioned, like
I think it would be possible to automatically generate it.
But before, I do need the domain - pinging @Michael Vogel - and a routing to the current
I'd like to generate it automatically during the CI, so the update would be smooth and automatically
Hi, I'd like to add a new domain `docs.friendi.ca` where we store the whole documentation. It should be versioned, like
https://docs.friendi.ca/2022.09/ for the docs of the stable 2022.09 version or https://docs.friendi.ca/develop/ for the docs of the develop branch.I think it would be possible to automatically generate it.
But before, I do need the domain - pinging @Michael Vogel - and a routing to the current
files.friendi.ca vm-instance - pinging @utzer ~Friendica~ I'd like to generate it automatically during the CI, so the update would be smooth and automatically
Roland Häder likes this.
Friendica Developers reshared this.
@Philipp Holzer @Michael Vogel @utzer ~Friendica~ Oh, even manuals are being updated and searchable. Now it is time to put links into #Friendica code. URL:
https://docs.friendi.ca/<branch>/ or only for develop?Philipp Holzer likes this.
Friendica Developers reshared this.
I don't have currently that time, so it's still WIP .. When it's finished, of course there will be a sub-dir for stable releases and develop - and Release candidates, similar to our branch structure :)
Roland Häder likes this.
Friendica Developers reshared this.
2 Factor & DB plaintext
!Friendica Developers
Hi all, I lost my mobile phone during the last festival and so my TOTP-App was gone forever. Unfortunately, I wasn't aware of my recovery codes, so I thought I would have to reset my accounts of Friendica.
But no! just use
@Hypolite Petovan , isn't this a possible security issue, is it?? Storing such sensible data as plaintext. I think we should save it as hash like for passwords to make it impossible to read it again .. Yes, the downside is that there's no possibility to save recovery_codes from the settings-panel again, but tbh I feel a little bit unsafe, but maybe it's just a feeling
Hi all, I lost my mobile phone during the last festival and so my TOTP-App was gone forever. Unfortunately, I wasn't aware of my recovery codes, so I thought I would have to reset my accounts of Friendica.
But no! just use
select code from 2fa_recorvery_codes where uid = 66 and used is NULL; and voilรก, I used the first code and was back in.@Hypolite Petovan , isn't this a possible security issue, is it?? Storing such sensible data as plaintext. I think we should save it as hash like for passwords to make it impossible to read it again .. Yes, the downside is that there's no possibility to save recovery_codes from the settings-panel again, but tbh I feel a little bit unsafe, but maybe it's just a feeling
Roland Häder likes this.
Friendica Developers reshared this.
We hash passwords not just because we want to prevent someone having a database dump from logging in the website depending on this database, but also for the other websites users might use the same password for. There is no such risk for randomly generated recovery codes.
Additionally, the use of the recovery codes necessitates the use of the password (that you still had in your head/password manager), after all it's a second-factor authentication, which means that it can be freely compromised as long as the first factor (the password) is safe.
Still, we can hash these codes and as a result only show them once, it isn't that hard other than requiring someoneโข to spend some quality time on this task.
Additionally, the use of the recovery codes necessitates the use of the password (that you still had in your head/password manager), after all it's a second-factor authentication, which means that it can be freely compromised as long as the first factor (the password) is safe.
Still, we can hash these codes and as a result only show them once, it isn't that hard other than requiring someoneโข to spend some quality time on this task.
Roland Häder likes this.
Friendica Admins reshared this.
Large database
!Friendica Admins
Hi everyone,
last week, I struggled with the size of my database for opensocial.at .. It went full (80GB) .. I added additional 10GB to the volume, but it seems to constantly increase and I think in a few weeks, I will be at the same situation like the last week.
Is there a possibility to shrink the database, or to wipe old data?
I recently deleted about 1.200 spam accounts (and set the register option to approval first ... lessons learned...). Are there leftovers inside the db which I can delete?
It's not that I don't have enough space, but I'm afraid of my backup borg-backup space in the near future ^^
Hi everyone,
last week, I struggled with the size of my database for opensocial.at .. It went full (80GB) .. I added additional 10GB to the volume, but it seems to constantly increase and I think in a few weeks, I will be at the same situation like the last week.
Is there a possibility to shrink the database, or to wipe old data?
I recently deleted about 1.200 spam accounts (and set the register option to approval first ... lessons learned...). Are there leftovers inside the db which I can delete?
It's not that I don't have enough space, but I'm afraid of my backup borg-backup space in the near future ^^
Friendica Admins reshared this.
nope, there're no active relays "Das System hat derzeit keinerlei Relays abonniert."
d778af0b0ee3462d6cff943e0a8d8a9a5364c67f
What I did:
What I did:
- Clicked at "Friendica Developers" under "Forums" inside the left navbar
- Opened a new Topic per pencil-button
Friendica Developers reshared this.
New Github/Gitea Label
!Friendica Developers
I'd like to introduce `Refactoring` as a new label. I feel like we should distinguish between "real" enhancements and "just" refactorings. What do you say?
I'd like to introduce `Refactoring` as a new label. I feel like we should distinguish between "real" enhancements and "just" refactorings. What do you say?
I feel like most current issue labels are meaningless beyond the Bug label we use to prioritize issues during the RC period.
So it wouldnโt do any harm to add another one, but it wonโt help with any existing workflow either.
So it wouldnโt do any harm to add another one, but it wonโt help with any existing workflow either.
Friendica Developers reshared this.
Locks with hostname
!Friendica Developers
Calling @Hypolite Petovan and @Michael Vogel for support at https://github.com/friendica/friendica/pull/10601
I do want to alter the way how locks are working to make parallel hosts/nodes possible. I think the logic itself is pretty straight forward and final.
BUT - how can I upgrade the lock table without using it during the update process itself, it's a gordon knot
I think about using a new method at
Do you think it works? At least my local nodes are working, but I'm totally unsure about it yet ..
I'd like to finish this PR for 2021.12 as another preparation for making Friendica "cluster ready" :D
Calling @Hypolite Petovan and @Michael Vogel for support at https://github.com/friendica/friendica/pull/10601
I do want to alter the way how locks are working to make parallel hosts/nodes possible. I think the logic itself is pretty straight forward and final.
BUT - how can I upgrade the lock table without using it during the update process itself, it's a gordon knot
I think about using a new method at
DBStructure: https://github.com/friendica/friendica/pull/10601/files#diff-eda5315231a6379d57916642e302e3525c3de2161e7dedb209526c7cfc8afcd6R543 and execute it before using the lock.Do you think it works? At least my local nodes are working, but I'm totally unsure about it yet ..
I'd like to finish this PR for 2021.12 as another preparation for making Friendica "cluster ready" :D
Add hostname to Locks by nupplaphil ยท Pull Request #10601 ยท friendica/friendica
Superseded #9114 This should help with #9095 We now add the hostname to each lock as well. The hostname is detected based on either uname -n or per environment variable NODE_NAME. This will have th...GitHub
This entry was edited (1 year ago)
Any idea @Michael Vogel ? :) I'm really stuck here and it's one major piece in the puzzle to make Friendica cluster-ready 
@Michael Vogel @Hypolite Petovan @Friendica Developers but wouldn't that possibly create a race condition, because a 2nd worker would start another update and wouldn't find the lock anymore so update it again?
Friendica Developers reshared this.
Questions Depository
!Friendica Developers
Calling @Hypolite Petovan for help
Why did you introduce a separate "Navigation" directory under "src"?
My guess: this is really a DDD practice using a directory for a use cases and define the different class-types under it (creating boundaries between differently purposed code). But if so, why don't you include the Modules/Notification as well?
And "Navigation/Notification" triggers for me, that the focus of this class is only the notifications at the top of the navbar, am I right?
I'm trying to follow your pattern and refactor the ProfileField / PermissionSet.
Would it be a new "src":
"src/Profile" , where PermissionSet and ProfileField are Entities/ValueObjects ? Or how would you structure these repositories/models?
Calling @Hypolite Petovan for help
Why did you introduce a separate "Navigation" directory under "src"?
My guess: this is really a DDD practice using a directory for a use cases and define the different class-types under it (creating boundaries between differently purposed code). But if so, why don't you include the Modules/Notification as well?
And "Navigation/Notification" triggers for me, that the focus of this class is only the notifications at the top of the navbar, am I right?
I'm trying to follow your pattern and refactor the ProfileField / PermissionSet.
Would it be a new "src":
"src/Profile" , where PermissionSet and ProfileField are Entities/ValueObjects ? Or how would you structure these repositories/models?
This entry was edited (1 year ago)
Aaahh sounds like the "good old" Model View Controller (MVC) pattern:
- Model --> where the business logic is (=> Entity, Depository, ...)
- View --> the presentation layer (=> Renderer / smarty templates)
- Controller --> The logic to show the right business logic (glue between model & view) (=> Modules)
=> And I like it
I'll give it a try after finishing the WebDav class
- Model --> where the business logic is (=> Entity, Depository, ...)
- View --> the presentation layer (=> Renderer / smarty templates)
- Controller --> The logic to show the right business logic (glue between model & view) (=> Modules)
=> And I like it
I'll give it a try after finishing the WebDav class
Friendica Admins reshared this.
Hypolite Petovan
•Friendica Admins reshared this.
Philipp Holzer
•@Hypolite Petovan @Philipp Holzer it's a XFS filesystem on a LVM
Hardware:
Roland Häder likes this.
Friendica Admins reshared this.
Hypolite Petovan
•Friendica Admins reshared this.
Philipp Holzer
•Hypolite Petovan likes this.
Friendica Admins reshared this.
Roland Hรคder
•xdebugand setmode=trace. Then use e.g.kcachegrindto analyze it, e.g. which method takes the most time. That might be a start! Yes, you normally want to havexdebugdisabled on live (no development) servers. But this is an exception as high load cannot be simulated properly to find the bottleneck.Philipp Holzer likes this.
Friendica Admins reshared this.
Michael Vogel
•'decoupled_receiver' => true,in your config. Then the inbox requests would be much faster. Then the processing is done via worker processes.like this
Roland Häder and Philipp Holzer like this.
Friendica Admins reshared this.
Michael Vogel
•But that one is tricky to set up. Best is to store the photos in a folder outside the web folder and then you can define a rule in your webserver that will redirect requests to the avatar folder to that folder.
Roland Häder likes this.
Friendica Admins reshared this.
Roland Hรคder
•Friendica Admins reshared this.
Philipp Holzer
•Roland Häder likes this.
Friendica Admins reshared this.
Michael Vogel
•Philipp Holzer likes this.
Friendica Admins reshared this.
Philipp Holzer
•Friendica Admins reshared this.
Michael Vogel
•Philipp Holzer likes this.
Friendica Admins reshared this.
Philipp Holzer
•/photo/contactroute for the avatar cache, don't I?Friendica Admins reshared this.
Michael Vogel
•No, the setup process is completely different. You have to have a path (somewhere) that is both readable and writable by the frontend and backend process
This path needs to be reachable when you call
https://your.server.tld/avatar(you have to redirect/avatar) or you have to defineavatar_cache_url.You you setup the file path from above in
avatar_cache_pathand setavatar_cacheto true. After you deactivate the caching of the avatars in the admin frontend, it should work.The URL path is stored in the contact table fields. So once you defined that URL, you mustn't change it again, since all old requests would fail. This mechanism is powerful, but currently too easy to misconfigure, that's why it is only accessible via the config file.
Friendica Admins reshared this.
Philipp Holzer
•Friendica Admins reshared this.
Michael Vogel
•like this
Philipp Holzer and Roland Häder like this.
Friendica Admins reshared this.
Roland Hรคder
•https://? That's never a good idea.Friendica Admins reshared this.
Philipp Holzer
•seems to work : https://friendica.philipp.info/avatar/4c/a1/d7c/53/8b/f08b/cd3c2904ac0bfd56-300.png?ts=1682970014
This image is now served by a completely different nginx than the frontend nginx . Because it's stateless, I'm now able to start more nginx for "just" serving stateless static content!
Brilliant!
Friendica Admins reshared this.
Michael Vogel
•Philipp Holzer likes this.
Friendica Admins reshared this.
Philipp Holzer
•Michael Vogel likes this.
Friendica Admins reshared this.
Philipp Holzer
•avatar/, there are still a lot of long running requests, see the image above. Do you have any idea? The overall CPU/RAM is on a normal state.